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Searching For Answers 



Shortly after the horrible 
shooting at Columbine High 
School in Littleton, Colo., I 
picked up The Universeand 
the Teacup: The Mathematics of Truth 
and Beauty, by K.C. Cole. In a chapter 
titled "Calculated Risks," Cole contrast- 
ed societal fears about airline safety 
whipped up in the aftermath of the 
mysterious crash of TWA Flight 800 
with society's seemingly resigned 
acceptance that thousandsof children 
(equivalent to dozens of fil led jumbo 
jets) dieevery day from malnutrition 
and disease around the world. She mar- 
vel ed at th e way we tu n e to th reats th at 
are "exotic, personal, erratic, and dra- 
matic." And, she noted, that doesn't 
mean we're ignorant, "just human." 

To many, videogames appear exactly 
that: exotic, dramatic, and yes, danger- 
ous. David Grossman, an Arkansas State 
University professor of military science, 
launched a crusade against violent 
videogames, and wants to hold them at 
least partially responsiblefor tragedies 
like those in Littleton and the 1997 
school shootings in Paducah, Ky. He 
believes that games like Quake not only 
influence children, they actually train 
them to shoot. "A hundred things can 
convince someoneto wantto takea 
gun and go kill," Grossman said in pre- 
pared testimony to a Sen ate committee 
thisspring. "But only onething makes 
them able to kill: practice, practice, 
practice." The implication here is that 
first-person action games provide skills 
necessary to pull off such a shooting. 

This is simplistic nonsense. It'stime 
for pundits I ike Grossman to quiet 
down until they can submit hard evi- 
dence to support such theories. Accord- 
ing to the New York Times, Mark Manes 
provided Eric Harris and Dylan Klebold 
with theTEC DC-9 used in thecrime, 
and the three of them practiced shoot- 
ing in theColorado mountainspriorto 
the incident. There is no evidence that 
any videogame trained thetwo killers to 
shoot. On the contrary, the facts seem 
to indicate that Harrisand Klebold had 
real practice firing their guns. 

As one peels away the layers of these 
cases and looks at the details as they 
emerge, onefindsthat these incidents 
are not as straightforward as people 
such asGrossman would like the public 
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to believe. As such, I believe that going 
after the producers of videogames, 
movies, and web sites in court will 
prove fruitless. Holding media corpora- 
tion s accou n tabi e for th e acts of men - 
tally ill minors (as 14-year-old Michael 
Carneal pled in the Paducah case) and 
adults on antidepressants (as 18-year- 
old Eric Harriswas) isno solution. 
Sadly, it'sjust another case of going 
after the deepest pockets 

Even if you discount any mental ill- 
nessinvolved, will thedeep pockets at 
companies such asActivision (oneof 
the defendants named in the Paducah 
case) have to shell out? Some have 
pointed to the recent legal victories over 
tobacco companies as proof that corpo- 
rationscan and should beliableforthe 
consequencesof their products But the 
gulf between the real dangers of ciga- 
rette smoking and the perceived dangers 
of playing videogames (or watching 
movies or surfing the web) is vast. 
Numerous independent medical studies 
have confirmed the link between smok- 
ing and cancer. Leaked internal docu- 
ments and testimony from former 
tobacco company employees acknowl- 
edged theculpability of tobacco firms 
and helped seal the fate of Big Tobacco. 
No such evidencelinks videogames and 
violent behavior. Thisisn't a simplecase 
of cause and effect. 

Lookingto thefuture, as games 
become more realistic and complicated, 
the pleasure of playing videogames may 
become harder to understand by those 
in society less attuned to thisform of 
entertainment. They will perceive 
videogames as a growing threat as poly- 
gon counts grow, color depth increases, 
and photo-realistic scenes become nor- 
mal. I don't think that videogames will 
be as fortunate as comic books, which 
were singled out in the 1950s as a harm- 
ful influence on children, and then 
gradually became accepted as an 
innocuous diversion. Videogames will 
always ridetechnology'sleading edge, 
and as such, they're apt to leave many 
paranoid conservatives in their wake. As 
an industry, we have to understand 
that, and be aware that we'll be under 
themicroscopefora longtime. ■ 



600 Harrison Street, San Francisco, CA 94107 

t: 415.905.2200 f: 415.905.2228 w: www.gdmag.com 

Publisher 

CynthiaA. Blair cblairgmfi.com 

EDITORiAL 

Editorial Director 

Aiex Dunne adunne(asirius.com 
Departments Editor 

Wesley Hail whailgsirius.com 
Editorial Assistant 

Jennifer Olsen joisencgimfl.com 
Art Director 

Laura Pool Ipooignifi.com 
Edit»r-At-Large 

Chris Hecker checker@d6.com 
Contributing Editors 

Jeff Lander jeffl@darwin3d.com 

Mei Guymon mel@surreal.com 

Omid Rahmat omid@compuservecom 
Advisory Board 

Hai Barwood LucasArts 

Noah Falstein Theinspiracy 

Brian Hook id Software 

Susan Lee-|Vlerrow Lucas Learning 

MarkMllier Harmonix 

Paul steed id Software 

Dan Teven Teven Consulting 

RobWyatt DreamWorks Interactive 

ADVERTiSiNG SALES 

Western Regional Sales VI anager 

Jennifer Orvik e: jorvlk@mfl.com t: 415.905.2156 
Eastern Regional Sales Manager/Recruitment 

Ayrlen Houchin e: ahouchm@mfl.com t: 415.905.2788 
International Sales Representative 

Breakout Marketing e: breakout_mktg@compuser\/e.com 

t: +49 431 801703 f:+49 431 801797 
ADVERTISING PRODUCTION 

SeniorVlce President/Production Andrew A. Mickus 

Advertising Production Coordinator Dave Perrotti 

Reprints Stella Valdez 1:916.983.6971 

M ILLER FREEMAN GAM E GROUP M ARKETING 
Group Marketing Manager GabeZlchermann 
MarComm Manager Susan McDonald 
Marketing Coordinator Izora Garcia de Lillard 

CIRCULATION 
Vice President/Circulation Jerry M . Okabe 
AssistantCirculation Director Sara DeCarlo 
Circulation Manager Stephanie Blake 
Circulation Assistant Kaushajackson-Crain 
Newsstand Analyst JoyceGorsuch 

INTERNATIONAL LICENSING INFORMATION 
RobertJ . Abramson and Associates Inc. 

t: 914.723.4700 f: 914.723.4722 
e: abramson@prodlgy.com 



Miller Freeman 

A United News & Media publication 

CEO/M iller Freeman Global Tony Tl 1 1 i n 
Chairman/MlilerFreeman Inc. Marshall W. Freeman 
President Donald A. Pazour 
Executive Vice Presidents Darrell Denny, Galen A. 
Poss, Reglna Starr Ridley 

Sr. Vice Presidents Annie Feldman, Howard I. Hauben, 
WinI D. Ragus, John Pearson, Andrew A. Mickus 
Sr. Vice President/Development Solutions Group KoAnn 
VIkoren 

Group President/Division SFl Reglna Ridley 

BPA 

MisuunoNU 

www.gdmag.com 



Backj 



SAYS 



YO U 



Does Sex Sell? 



■ find your magazine to have way too 
many sexual connotations. Are 
tliese adult games only or do they mar- 
ket to children, too? Advertisements on 
pages 18, 19, 28 and 63 (April 1999) are 
very inappropriate. I understand that 
sex sells, but isn't there another way 
companies can think of to make a buck 
and save our society and children from 
negative, wasted energy? Basically, I 
was disgusted by the severity of sexual- 
ity towards women . 

Jennifer Dennis 
via e-mail 

I must protest the increasing number 
of advertisements that feature scant- 
ily-clad women and sexual innuendo 
appearing in Game Developer. Looking 
back over the last several issues, the 
number seems to havejumped sharply 
in April. 

The game industry has long had a 
reputation as a boys' club which ishos- 
tileto women. This reputation has 
compromised our ability to attract top 
female talent to our companies, and 
therefore, to make the best games pos- 
sible. Many women developers are 
uncomfortable with the blatant appeals 
to sexuality which are the stock-in - 
tradeof our industry's advertising. This 
material now seems to be creeping 
from the gamers' magazines into our 
own trade journals, and it sends a dis- 
tinct message that women are not 
wanted or welcome in this business. I 
trust this is not a message that Game 
Developer agrees with. 

Th ese ad verti semen ts are i n su I ti n g to 
our intelligence as developers. No 
developer is so foolish as to make tech- 
nical purchasing decisions on the basis 
of whose adsfeaturenude women; and 
to assumethat we would isto charac- 
terize us as oversexed and stupid. 

Asa longtime subscriber to a variety 
of tradejournals, I assure you that such 
material is neither ordinary nor appro- 
priate for them. It certainly does not 
appear in Electronic Engineering Times, 
for example, even though its reader- 
ship, I ike Game Developer's, isalso pre- 
dominantly male. I urge you to reject 
any more advertising on these themes. 

Ernest W. Adams 
Electronic Arts 
Redwood City, Calif. 




Really, We're Just Friends 

Your article "Dolby and Aureal: 
Contrasts in Audio" (Hard 
Targets, May 1999) was interesting 
and insightful, and 1 thank you for 
writing it. 

I am writing to correct you on one 
point you made near the end of the 
article: "Dolby then has to look at DTS, 
and George Lucas's THX among other 
competitors." The point I wish to cor- 
rect is regarding THX as 
a competitor to 
Dolby. This is a 
common mis- 
conception that 
THX and Dolby 
are competing 
technologies, when 
in fact they comple- 
ment each other. 

THXisa 
hardware certifi- 
cation pro- 
gram to 
en sure the 
best sound 
reproduc- 
tion, in 
other words, 
to reproduce 
it as close as 
possible to the 
original. Dolby SR, Dolby Digital, DTS, 
and SDDS are sound formats that, if 
reproduced on the highest-quality 
THX-certified systems, will produce an 
unsurpassed quality of audio. You are 
correct when you state DTS as compe- 
tition to Dolby. 

Paul Widner 
via e-mail 



Let Him Who Has Played Sin 
Cast the First Stone 



■ had particular interest in your 
Postmortem on Ritual Entertain- 
ment'sSiN (March 1999). Indeed, the 
column is my favorite in the magazine, 
and I was well aware of the horrible 
bugsin theinitial release of 9n. 

In reading the article, I wasa little 
shocked that so little mention was 
made of the bugs on release. At the 
very least, they should have been 
under one of the "Things That Went 
Wrong." There are a few issues sur- 




rounding author Scott Al den's claims 
that disturb me. 

First, just how bad istheir quality 
assurance group? By some reports, the 
game took approximately five minutes 
to load both between levels and when 
restori n g a saved game. I f th e game was 
run from theCD auto-play screen, it 
ran from the CD instead of the hard 
drive, causing save game troubles. 
Saved games themselves were enor- 
mous— people were losing hundreds 
of megabytes from them! There 
was also a bug with the 
first boss encountered, 
which ruined the 
encounter entirely for 
many people. 
These bugs simply 
"slipped through"? 
Second, there was mention of 
the patch, and Acti vision distributing 
freeCDsto all who requested them. I 
do not feel that was an act of charity, 
but an attempt to right a wrong. Some 
of the newsgroup chatter hinted at in 
thearticlementioned an apparent 
release agreement with Activision. If 
the game was not out by a certain date. 
Ritual was to suffer a rather large eco- 
nomic penalty. (Presumably, the inten- 
tion was to beat Half-Life to market.) 
Hence, the game shipped early with 
known bugs. 

Hearsay? Conjecture? Or swept 
under the carpet? 

Jose Fernandez 
via e-mail 

RITUAL'S LEVELORD RESPONDS: 

First, you must realize that no game devel- 
oper ever releases bugs intentionally. In the 
case of Sin, we worked very long and hard 
for almost two years. It seemed to be an 
endless crunch time and I nearly had a ner- 
vous breakdown from the long-term stress 
and time demands. I know this seems like a 
dream job, and it is, but I truly almost lost 
my mind from Sin. Losing your mind, by the 
way, is not fun like some acid trip — it's 
very scary! 

Anyhow, you must know that we wouldn't 
just throw something out after all that work. 
If it means anything to say this, we are 
heartbroken that such a cool game as Sin is 
now used as a symbol of what not to do. 
There are other reasons for Sin's demise. 

Similar to proofreading your own writing 
after you've already read it a thousand times, 
or taste-testing a meal after you've been in 
the kitchen all day nibbling the ingredients 
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and smelling the seasonings, you really can 't 
effectively and thoroughly verify your own 
worl<. You start to see the forest and not the 
trees, and you mal<e too many assumptions 
in the fog of familiarity. You need new eyes 
to truly test anything. We were counting on 
our publisher to do this. 

Being a small development team com- 
pared to most out there, we also lacked 
the person-power to perform complete 
testing. We were counting on our publish- 
er to supplement this need. Each tribe 
member was hurriedly rushing right up 
until the very last moment. Burned out and 
fatigued, it was like trying to herd a dozen 
cats after running a marathon. Remember 
the scene in Jaws in which Richard 
Dreyfuss is frantically tying a buoy to the 
spear gun line and yelling at Robert 
Shawn, "Don't wait for me!" as he aimed at 
the approaching shark? That's what it was 
like at the end of Sin. That's what the end 
of most games is like. 

We were assured that a large team of 
testers would verify the game and make 
sure it was valid. When we went beta, we 
indeed got a large load of bugs and misbe- 
haviors. This list, however, never seemed 
to change much other than to get smaller 
as we checked off each one. This seems 
obvious now and a warning flag should 
have gone up, but we were too concentrat- 
ed on finishing and burned out from 
exhaustion. It all ended with a "No show- 
stoppers! We're going gold!" from the pub- 
lisher. We assumed. 

Being small also means that we do not 
have all the varieties of test platforms 
(sound cards, video cards, and so on) to 
test across the board. We also tend to 
develop on our network where slow load 
times are the norm due to pipeline traffic. 
We simply do not have the time to burn CDs 
and load the game on isolated platforms, 
especially towards the end. We were count- 
ing on our publisher to do the hardware- 
related testing, too. 

Please know that we tried very, very hard 
to make Sin the coolest game ofiggS. It 
was the coolest, if not a good second 
place. Also know that we did everything 
possible to debug the final version and test 
every nuance and permutation of game 
play. Finally, know that no one is hurt more 
by this rush-to-the-shelves calamity than 
we are. After having the privilege of work- 
ing on such excellent games as Duke Nukem 
3D and The Scourge ofArmagon, it was a 
tremendous blow that Sin was not to follow 
suit merely for reasons of bad management 
and handling. 



Should the Hall of Fame 
Be the Hall of Games? 



It's revealing sometimes to glimpse 
at what makes people do the 
things they do. Often it's a complex 
combination of physical, psychologi- 
cal, and yes, spiritual impulses. For 
some reason, we in the entertainment 
industry seem to want to reveal those 
driving influences more than the 
average person does. Maybe it's 
because we're in the business of hav- 
ing fun and we think that others 
might want to know why we are in 
this crazy business. Sometimes they 
really do want to know. But I believe 
that mostly they don't. 

It's with these thoughts in mind that 
I consider Ernest Adams's notion of a 
Computer Game Hall of Fame 
("I mmortality for Game Developers," 
Soapbox, April 1999). Whilethe con- 
cept of a Hall of Fame is intriguing, he 
told me way more than I wanted to 
know. For Ernest, it's all about immor- 
tality. He wants his own pyramid. Well 
I have news for you, Ernest. The pyra- 
mids may seem immortal, but the 
pharaohs never do. 

Computer gaming is a relatively 
young endeavor compared to other 
pursuits. I I ike the fact that we don't 
have a lot of self-appointed experts in 
tweed coats rubbing their chins and 
harrumphing about this or that 
"important" game in the history of 
games. Adams states that the Hal I of 
Fame "would be a place where the 
great games are kept, and talked 
about, and studied for the wonder and 
truth that they contain. Above all, it 
would be a place where their designers 
are honored." Wonder and truth? The 
fact is that the games we make are fun 
and technically amazing and yes, 
worthwhile. But let us not get carried 
away and overstate our contribution 
to mankind. 

Having a place where we can seethe 
development of computer games — to 
seethe history of computer gaming 
including thedesigners— isa good 
and interesting idea. A place where we 
mostly raise monuments to the 
buildersisless satisfying. I understand 
that M r. Adams lost a friend and I 
understand wanting to honor that 
friend. I also understand the need for 
recognition and love. But I would pre- 



fer to honor the achievement much 
more than the achievers. After all, as 
Shakespeare might have said, the 
game's the thing. 

Glenn O'Bannon 
Rainbow Studios 
via e-mail 

In response to Ernest Adams's 
Soapbox column "Immortality for 
Game Developers" (April 1999), com- 
puter game museums do exist. Look up 
http://www.computerspielemuseum.de 
and http://www.trans-japan.com/ 
vp/bg96. 

But the idea is nonsense. Art that 
cannot be experienced is void. We 
don't need a museum; we need good 
emulators, and we need the greed- 
head companies that crackdown on 
them to figure out a way to let them 
thrive. Dani Bunten will be remem- 
bered as more than a marginal figure 
only if future generations can experi- 
ence her games. 

Greg Costikyan 
via e-mail 



ERNEST ADAMS RESPONDS: 

"L'homme n'est rien; c'est I'oeuvre qui est 
tout," as Gustave Flaubert observed to 
George Sand. ("The man is nothing; the 
work is everything. ") Ignoring the author in 
favor of the work is the modus operandi of 
oh-so-trendy postmodern literary criticism, 
and Glenn O'Bannon — his derision for 
intellectuals aside — places himself square- 
ly in that camp. Call me old-fashioned, then, 
but I must disagree. To honor the artist 
along with the art is no more than simple 
justice, as Greg Costikyan eloquently argues 
in his online essay on the subject, which can 
be found at http://www.crossover.com/ 
costikjjustice.html. Should we remember 
Shakespeare's plays, but forget 
Shakespeare? Remember Mozart's music, 
but forget Mozart? Remember Spielberg's 
movies, but forget Spielberg? That would be 
cruelty, indeed. 

If those figures are too grand, then try 
considering my original model, the Pro 
Football Hall of Fame. Can we remember 
Walter Payton's running without remem- 
bering Walter Payton, or Dick Butkus's 
tackling without remembering Dick 
Butkus? Should we try? The best computer 
games are expressions of a guiding vision. 
Let us then praise the visionaries along 
with their work. 
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P New Products 

btf Alex DM*t*te 

SurfaceSuite Pro 1.5 

SVEN TECHNOLOGIES released version 
1.5 of SurfaceSuite Pro, the company's 
3D texture mapping software which 
lets you apply 2D Images to 3D polyg- 
onal, NURBS and patch models. 
Version 1.5 adds two significant new 
features to the product. The first, 
AlphaPaint, lets you paint alpha masks 
onto your textured model, giving you 
more control over texture blending. 
The second addition isPatchWork, 
which lets you create a "quilt" of tex- 
tures — a single skin — for real-time 
polygonal models that you can export 
to your game engine. 

The new version also supports 
Softimage's .HRC and Rhino's 3D 
model formats, as well asthe.PCX 
Image format. Sven perked up the 
product's Interface by adding multiple 
keyboard shortcuts and hotkeys, float- 



ing windows, automatic layer cre- 
ation, and better support for Import- 
ing materials. 

SurfaceSuite Pro supports 3D Studio 
Max, Softimage, Maya, Lightwave 3D, 
and Rhino, among other 3D modeling/ 
animation environments. The stand- 
alone product Is priced at $595, and 
upgrade discounts are offered. 
■ Sven Technologies 

Palo Alto, Calif. 

(650) 852-9242 

http://www.sven-tech.com 



Maya Z 




ALIASIWAVEFRONT introduced Maya 2, 
the latest version of Its 3D animation 
and visual effects tool. Perhaps most 
Important to game developers, the 
company Improved the modeling tools 
— you now have more precise control 
over curve and surface geometry, new 
toolsfor smoothing polygonal surfaces, 
new texturing tools and the ability to 
assign arbitrary data to polygon ver- 
tices. With the Maya Unlimited pack- 
age, you get more advanced modeling 
tools (Including sup- 
port for subdivision 
surfaces) as well as fur 
and cloth animation 
features. 

In the area of char- 
acter modeling and 
animation, Maya 2 
has new deformer 
types and automated 
skinning capabilities 
for faster creation and 
easier control of com- 
plex characters. The 
tool uses a pose-based 
approach that lets 
you treat a complex 
character as a single 
entity as you animate 
it. Allas|Wavefront 
also souped up 
Maya's renderer. 



New Products: Sven Technologies 
updates SurfaceSuite Pro, Alias|Wave- 
front rolls out Maya 2, and OpenGL 
arrives for Macintosh, p. 9 



Industry Watch: Sega reveals pricing 
for Dreamcast, EA hits the billion-dollar 
mark, Activision squeaks by, and 
Interplay faces red ink. p. 10 



which can speed rendering up by 90 
percent in some cases, according to the 
company. Maya 2 also supports multi- 
threaded batch rendering, too. 

Maya Complete 2 is priced at $7,500 
and Includes modeling, rendering, ani- 
mation, dynamics. Artisan and MEL 
(the tool's embedded scripting lan- 
guage) features. Maya Unlimited 2 
costs $16,000 and includes the features 
found in the Complete version, plus 
Maya Live, Maya Fur, Maya Cloth and 
new advanced modeling features. Both 
M aya flavors are available for Windows 
NT and IRIX. 
■ AliaslWavefront 

Toronto, Ont., Canada 

(416) 362-9181 

http://www.aw.sgi.com/entertainment 



OpenGL for the Macintosh 

APPLE recently shipped OpenGL for 
Macintosh, further illustrating the way 
in which Stevejobs is shifting Apple's 




focus back towards games. Available 
freely from the Apple web site for 
download, OpenGL for Macintosh 
brings the Industry-standard 3D API to 
the Mac. OpenGL for Macintosh 
requires a PowerPC -based Macintosh 
computer with M acOS 8.1 or later An 
I Mac or new Power Macintosh G3 is rec- 
ommended for accelerated 3D render- 
ing. The version available from Apple's 
web site includes libraries to accelerate 
rendering on Rage II, Rage Pro, and 
Rage 128-based Macintosh systems. 
■ Apple Computer 

Cupertino, Calif. 

(408) 996-1010 

http://www.apple.com/opengl 



Maya 2's enhancements promise game developers 
improved animation features, including new fur render- 
ing, as seen on ttiis tiger. 
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DISNEY INTERACTIVE AND NINTENDO 

announced a new partnership, which 
will introduce M ickey M ouse to the 
world of 3D games. Nintendo will 
publish a number of Mickey Mouse- 
based games for the N 64 and the 
Game Boy Color, to be developed by 
Rare Ltd. Two of the titles will be rac- 
ing games for the N64 and Game Boy 
Color (coming out in 1999 and 2000), 
and the third will be a "Mickey 
Adventure" title, slated for release in 
2001 on Nintendo's next console sys- 
tem and the Game Boy Color. 
Additionally, as part of the agreement, 
Disney Interactive will develop multi- 
ple titles for the Game Boy Color 
aimed at the girl -games market. These 
titles will be based upon Disney's 
Beauty and the Beast and Alice in 
Wonderland movies. 

SPEAKING OF THE GAME BOY COLOR, 

Nintendo said that sales of the Game 
Boy Color during the first quarter of 



UPCOMING EVENTS 

CALENDAR 



Macworld Expo 



JACOB J. JAVITS 
CONVENTION CENTER 

New York, NY 
July 21-23, 1999 
Cost: $45-$l,195 
http://www.macworldexpo.com 



Slggroph '99 



LOS ANGELES CONVENTION CENTER 

Los Angeles, Calif. 
August 8-13, 1999 
Cost: $25-$760 
http://www.siggraph.org/s99 



1999 averaged 94,000 units per 
week, compared to 31,000 in 
1998 for the previous Game Boy 
unit. Peter Main, Nintendo's 
executive vice president of sales 
and marketing, said that since 
the launch of the Game Boy 
Color in the U.S. last November, 
the company has sold more 
than two million units. 

ATOMIC POWER AT MINDSCAPE. 

M indscape signed a publishing 
deal with Atomic Games, which 
lets the 'Scape take over the 
successful Close Combat series 
from former publisher 
Microsoft. The deal adds another 
strong war game franchise to M ind- 
scape's lineup, which already included 
Panzer General and Steel Panthers. The 
next game scheduled for the Close 
Combat series is slated for release 
sometime in the fourth quarter. 

ACTIVISION BARELY BEATS ESTIMATES. 

Activision's fourth fiscal-quarter (ending 
M arch 31) profits were a bit better than 
analysts had expected, amounting to 
$5.2 million, which was an improve- 
ment over the $689,000 that the com- 
pany earned last year during the same 
period. CEO Bobby Kotick credited the 
company's wide array of games for 
boosting the company's market posi- 
tion, and indicated that the company 
was on the prowl for large acquisition 
targets However, CFO Barry Plaga pre- 
dicted Activision would post a loss in 
the first fiscal quarter of 1999, as it did 
last year. 

ARCADE DOWSER. In an attempt to 
counteract the shrinking arcade market 
and make it easier to bag that increas- 
ingly elusive quarry, the loyal arcade 
gamer, Midway Games and WMS 
Industries launched a web-based data- 
base application that tells you exactly 
where you can find Midway coin -op 
games around the world. This simple 
game finder, available on on Midway's 
website (http://www.midway.com), also 
lets visitors submit new locations when 
they find games. Planning a trip to 
Madagascar? Need to satisfy your 
CarnEvil fix while you're there? Now 
you know where go... 

MUSIC TO THEIR EARS? Aureal Semi- 
conductor, maker of the Vortex2 digital 




Mindscape is adding tlie Close Combat series to 
its arsenal of war games. 



audio processor and theA3D audio 
API, announced financial results for 
the first fiscal quarter ending April 4, 
1999. Revenues reached $12.6 million, 
amounting to a 250 percent increase, 
but saw a net loss of $4.1 million (bet- 
ter than its $5.5 million loss last year). 
What looks most promising is that the 
company's gross margins grew to 34 
percent in the first quarter from 22 
percent last year, thanks to increased 
demand for sound cards based on the 
Vortex 2. 

DREAMCAST PRICING AT LAST. Under- 
cutting industry estimates, Sega's Bernie 
Stolar revealed that the the Dream cast 
suggested launch price would be $199. 
Additionally, Stolar confirmed that the 
U.S. launch date would be September 9, 
1999. There will be between 10 and 12 
titles available around launch time, 
plus more than 20 first-party titles on 
track for the year 2000. To date, retail- 
ers have placed pre-orders for 30,000 
systems, according to Sega. 

$io9. EA released its fiscal year results. 
Two words: banner yean Congratula- 
tions go out to Larry Probst and com- 
pany, for being thefirst entertainment 
software publisher to top a billion dol- 
lars in revenue. 

NEWS FROM THE LAND OF RED INK. 

Poor Interplay continues to have a 
tough time. The company lost $8.28 
million in its first quarter, compared to 
a profit of $3.1 million a year ago. 
Interplay blamed its I arger-th an -expect- 
ed loss on the fact that it hasn't 
released any major titles recently, and 
the impact of higher-th an -anticipated 
product returns ■ 
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Flex Your Facial 
Animation Muscles 




ast month I left off with a nice short list of the vi semes I would need to 
represent speech realistically. However, now I am left with the not 
insignificant problem of determining exactly how to display these 
vi semes in a real-time application. 



It may seem as if this is purely an art 
problem, better left to your art staff (see 
Artist'sView this month, "Talking 
Heads: Hierarchical Animation in Real- 
time 3D"). Or, if you are a one-person 
development team, at least left to the 
creative side of your brain. However, 
your analytical side needs to inject itself 
in herea bit. Thisisoneof thoseearly 
production decisionsyou read about so 
much in the Postmortem column that 
can make or break your schedule and 
budget. Choose wisely and everything 
will work out great. Choose poorly and 
your art staff or even your own brain 
will throttle you. 

Decisions, Decisions 

For the final result, I want a 3D real- 
time character that can deliver vari- 
ous pieces of dialog in themost con- 
vincing manner possible. Thanks to the 
information learned last month, I know 
I can severely limit the amount of work 
I need to do. I know that with 13 
visemes, or visual phoneme positions, I 
can reasonably represent most sounds I 
expect to encounter. I even have a nice 
mapping from American English to my 
set of vi semes. Mostotherlangu ages 
could probably be represented by these 
visemes as well, but could requirea dif- 
ferent mapping table. 

From this information I can expect 
that if I can reasonably represent these 
13 visemes with my character mesh. 



then continuous lip-synch should be 
possible. So the problem really comes 
down to how I construct and manipu- 
late those meshes. 



Viseme-Dosed Metliods 

Certai n ly, th e obvi ous meth od for 
creati n g th ese 13 vi semes i s to gen - 
erate 13 versions of my character head 
mesh, oneto represent each viseme. I 
can then use the morphing techniques I 
discussed in last December's column 
("Mighty Morphing Mesh Machine," 
December 1998) to interpolate smooth- 
ly between different sounds. 

Modeling the face to match the 
visemes is pretty easy. Once the artist 
hasthe base mesh created, each viseme 
can be generated by deforming the 
mesh any way necessary to get the 
right target frame. As long as no ver- 
tices are added or deleted and thetri an- 
gle topology remains the same, every- 
thing should work out great. Figure 1 
shows an image of a character display- 
ing the "L" viseme, as in the word 
"life." The tongue is behind the top 
teeth, slightly cupped, leaving gaps at 
thesideof the mouth, and theteeth 
are slightly parted. 

Sounds pretty good so far. J ust create 
13 morph targets for the visemes in 
addition to the base frame and you're 
done. Life's great, back to physics, 
right? Well, not quite yet. 
Suppose in addition to simply lip- 



W hen not massaging thefaces of digital beauties or doing stunt falls in a mo-cap 
rig, Jeff can be found flapping his own lips at Dara/in 3D. Send him some snappier 
dialogueatjeffl@darwin3d.com. 
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synching dialog, your characters must 
express some emotion. You want them 
to be able to say things sadly, or speak 
cheerfully. We need to add an emo- 
tional component to the system. 



Adding Some Heart to tlie Story 

At first glance, it may seem that 
you can simply add some addi- 
tional morph targets for the base emo- 
tions. Most peopledescribesix basic 
emotions. Here they are with some of 
their traits. (SeeGoldfinger under "For 
Further Info" for photo examples of 
the six emotions.) 

1. Happiness: Mouth smiles open or 
closed, cheeks puff, eyes narrow. 

2. Sadness: Mouth cornserspull 
down, brows incline, upper eyelids 
droop. 

3. Surprise: Brows raise up and arch, 
upper eyelids raise, jaw drops. 

4. Fear: Brows raise and draw togeth- 
er, upper eyelids raise, lower eyelids 
tense upwards, jaw drops, mouth cor- 
ners go out and down. 

5. Anger: Inner brows pull together 
and down, upper eyelids raise, nostrils 
may flare, lips are closed tightly or 
open exposing teeth. 

6. Disgust: M iddle portion of upper 
lip pulls up exposing teeth, inner brows 
pull together and down, nose wrinkles. 

Th ere are vari ati on s of th ese em o- 
tions, such as contempt, pain, distress, 
excitement, but you get the idea. Very 
distinct versions of these six will get 
the message across. 

Thekey thingto notice about this 
list isthat many of these emotions 
directly affect the same regions of the 
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FIGURE 1. The "I" viseme as seen at 
the start of the word "life. " 



model as thevi semes. If you simply 
layer these emotions on top of the 
existing viseme morph targets, you can 
get an additive effect. This can lead to 
ugly results. 

For example, let me start with the 
"L" sound from before and blend in a 
surprised emotion at 100 percent. The 
"L" sound moves thetongue up to the 
top set of teeth and parts the mouth 
slightly. However, the surprise target 
drops thejaw even farther but leaves 
the tongue alone. Thiscomblnation 
blends into the odd-looking character 
you see In Figure 2. 

This problem really becomes appar- 
ent when the two meshes are actually 
fighting each other. For example, the 
"oo" viseme drives the lips into a tight, 
pursed shape whilethesurprise emo- 
tion drivesthe lips apart. Nothing pret- 
ty or realistic will come out of that 
combination. 

When I ran into this Issue a couple 
of years ago, the solution was tied to 
the weighting. By assigning a weight or 
priority to each morph target, I can 
com pen sate for these problems I give 
the "oo" viseme priority over the sur- 
prise frame. Thiswill suppressthe 
effect that the surprise emotion has 
over shared vertices. 



Welcome to Muscle Beach 

ost of the academic research on 
facial animation has not 
approached the problem from a viseme 
basis Thislsdueto a fundamental 
d rawback to th e vl seme frame based 
approach. In the vi seme-based system, 
every source frame of animation is 
completely specified. While I can speci- 
fy the amount each frame contributes 
tothefinal model, I cannot create new 



FIGURE 2 . A very surprised "I" 
viseme. 



source models dynamically. Say, for 
example, I want to allow thecharacter 
to raise one eyebrow. With the frames I 
have described so far, this would not be 
possible. In order to accomplish this 
goal, I would need to create Individual 
morph targets with each eyebrow 
raised individually. Si nee a viseme can 
Incorporate a combination of many 
facial actions. Isolating these actions 
can lead to an explosive need for 
source meshes. You may find yourself 
breaking these targets into isolated 
regions of the face. 

For this reason, researchers such as 
Frederic Parke and Keith Waters began 
examining 



System (FACS), describes 50 of these 
action units that can create thousands 
of facial expressions By creating a facial 
model that is control led via these action 
units. Waters was able to simulate the 
effect that changes in theactlon units 
reveal on the skin. 

Whilel'm not sure If artists are ready 
to start creating parametric models 
controlled by virtual muscles, there are 
definitely some lessonsto be learned 
here. With this system, it's posslbleto 
describe any facial expression using 
these 50 parameters. It also completely 
avoids the additive morph problem I 
ran into with the viseme system. Once 
a muscle is completely contracted. It 
cannot contract any further. This limits 
the expression to ones that are at least 
physically possible. 

Artist-Driven Muscle-Based Facial 
Animation 

Animation toolsarenot really 
developed to a point where artists 
can place virtual muscles and attach 
them to a model. This would require a 
serlouscustom application that the 
artists may be reluctant even to use. 



how the face 
actually works 
biologically. 
By examining 
themuscle 
structure 
underneath 
theskin, a 
parametric 
representation 
of the face 
became possi- 
ble. In fact, 
psychologists 
Paul Ekman 
and Walllce 
Friesden devel- 
oped a system 
to determine 
emotional 
state based on 
the measure- 
ment of Indi- 
vidual muscle 
groups as 
"action units" 
Their system, 
called Facial 
Action Coding 



Action 


Muscle Name 


Effect 


Raise Inside Brow L/R 


Frontalis Medial portion 


Many Expressions 


Raise Outside Brow L/R 


Frontalis Lateral portion 


Many Expressions 


Tighten Inside 
Brow Frown 


Corrugator Supercilii 
+ Procerus 


Anger, Pain, Disgust 


Eyes Wide L/R 


Levator Palpebrae Superioris 


Surprise, Fear, Shock 


Eye Squint L/R 


Orbicularis Oculi 
orbital portion 


Anger, Thought, 
Concentration 


Eyelid Close L/R 


Orbicularis Oculi palpebral portion 


Blink, Wink 


Nostril Flare L/R 


Dilator Naris + Levator Labii 
Superioris Alaeque Nasi 


Disgust 


Purse Lips 


Incisivus Labii 


Kiss, Anger, "oo", 
Whistle 


Smile Corner L/R 


Zygomaticus l\/lajor 


Smile 


Corner mouth down 
Into Sadness L/R 


Depressor Anguli Oris + Zygomaticus 
Minor + Depressor Anguli Oris + 
Mentalis 


Sadness 


Top Up Up L/R 


Levator Labii Superioris 


Disgust, 

Part Lips for Sounds 


Lower Lip Down L/R 


Depressor Labii Inferioris 


Part Lips for Sounds 


Tighten Lips U/L 


Orbicularis Oris 


" p " , " b " , " m " , 
Anger 


Jaw Open 


Digastric 


Speaking, Surprise 


jaw Slide L/R 


Masseter 


Slide law L/R 


CHART 1 . The basic muscle groups involved in facial animation. 
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FIGURE 3. rfte zygomaticus major 
muscle will put a smile on your face. 



However, that doesn't mean that these 
meth ods are n ot avai I abl e for game 
production. It just requires a different 
way of thinl<ing about modeling. 

For instance, let metakea look at 
creating a simple smile. Biologically, I 
smile by contracting thezygomaticus 
major muscle on each side of my face. 
Th i s m usci e con n ects th e outsi de of th e 
zygomatic boneto the corner of the 
mouth as shown in Figures. Contract 
onemuscleand half a smile is born. 

O.K. Mr. Science, what does that 
have to do with modeling? Well, this 
muscle contracts in alinear fashion. 
Take a neutral mouth and deform it as 
you would when the left zygomaticus 
major is contracted. This mesh can be 
used to createadelta tableforall ver- 
tices that change. Repeat this process 
for all the muscles you wish to simulate 
and you have all thedatayou need to 
start making faces. You will find that 
you probably don't need all 50 muscle 
groups described in the FACS system. 
Particularly if your model has a low 
polygon count, this will be overkill. 
The point isto create the muscle 
frames necessary to create all the 
visemesand emotionsyou will need, 
plus any additional flexibility you 
want. You will probably want to add 
some eye blinks, perhaps some eye 
shifts, and tongue movement to make 
the simulation more realistic. 

The FACS system is a scientifically- 
based general modeling system. It does 
not consider the individual features of 
a particular model. By allowing the 
modeler to deform the mesh for the 
muscles instead of using thisalgorith- 
mic system, I am giving up general 
flexibility over a variety of meshes. 



FIGURE 4. Pjyc/cer up: Incisivus labii 
at worl<. 



However, I gain creative control by 
allowing for exaggeration as well as 
artistic judgement. 

Thedownside isthat it isnow much 
harder to describe to the artists what it 
isyou need. You need to purchase 
some sort of anatomy book (see my 
suggestions at the end of thecolumn) 
and figure out exactly what you want 
to achieve. Your artists are going to 
resist. You had this nice list of 13 
visemesand now you are creating more 
work. They don't know what an inci- 
sivius labii isand don't want to. You 
can explain that it is what makes Lara 
pucker up and they won't care. You 
will have to win the staff over by show- 
in g th e creative possi bi I i ti es f or ch arac- 
ter expression that are now available. 
They probably still won't care, so get 
the producer to force them to do it. I 
have created a sample muscle set in 
Chart 1. Thiswill give you somegroups 
from which to pick. 

Now I need to relate these individual 
muscle meshes to thevisemeand emo- 
tional states. This is accomplished with 
"muscle 
macros" that 
blend the per- 
centages of the 
basic muscles to 
form complex 
expressions 
Thisflexibility 
permits speech 
and emotion in 
any language 
without the 
need for special 
meshes. 

I still need to 
handlethecase 



where several muscles interact with the 
same vertices. However, now there isa 
biological foundation to what you are 
doing. 

Certain muscles counteract the 
actions of other muscles. For example, 
the muscles needed to create the "oo" 
viseme (incisivius labii) will counter the 
effect of thejaw dropping (digastric for 
those of you playing along at home). 
One real-time animation package! 
have been working with called 
Geppetto, from Ouantumworks, calls 
this M uscle Relations Channels. You 
can create a simple mathematical 
expression between the two to enforce 
this relationship. You can see this 
effect in Figure 5. 



Now for the Animation 

■ finally have my system set up and 
my models created. Itistimeto cre- 
ate some real -time animation. The 
time-tested animation production 
method isto take a track of audio dia- 
log and go through it, matching the 
visemesin your model set to the dia- 
log. Then, in second pass, go through it 
and add any emotional elements you 
want. This, as you can imagine, is pret- 
ty time consuming. Complicating the 
matter is that there are not many off- 
the-shelf solutionsto help you out. The 
job requires handling data in a very 
special way and most commercial ani- 
mation packages are not up to thetask 
without help. 

Detecting the individual phonemes 
within an audio track is part of the 
puzzle that you can get help with. 
There isan excellent animation utility 
called Magpie Pro from Third Wish 
Software that simplifies this task. It can 




FIGURE 5. W.C. l-ields's jaw is open and then blended with the 
"oo" viseme. Image courtesy of Virtual Celebrities Productions 
and Ouantumworks. 
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take an audio track and analyze it for 
phoneme patterns you provide auto- 
matically. While not entirely accurate, 
it will at least get you started. From 
there you can manually match up the 
visemesto the waveform until it looks 
right. The software also allows you to 
create additional channelsfor things 
such as emotions and eye movements. 
All this information can be exported as 
a text file containing the transition 
information. This in turn can be con- 
verted directly to a game-ready stream 
of data. You can see Magpie Pro in 
action in Figure 6. 



Wire Me Up, Baby 



With all the high-tech toys avail- 
able these days, it may seem 
likea waste to spend all thistime 
hand-synching dialog. What about 
this performance capture everyone has 
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Faciol Animation 

http://mambo.ucsc.edu/psl/fan.html 

Gesture Recognition 

http://www.cs.cmu.edu/~face 

Performance Animation Society 

http://www.pasociety.org 

Magpie Pro 

http://thirdwlsh.simplenet.com 

Filmbox 

http:/ /www.kaydara.com 

Geppetto 

http://www.quantumworks.com 
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been talking 
about? There 
are many facial 
capture devices 
on the market. 
Some determine 
facial move- 
ments by look- 
ing at dots 
placed on the 
subject's face. 
Others use a 
video analysis 
method for 
determining 
facial position. 
For more 
detailed infor- 
mation on this 
aspect, have a 
look at Jake 

Rodgers's arti cl e " An i mati n g Faci al 
Expressions" (November 1998). The 
end result is a series of vectors that 
describe how certain points on the 
face move during a capture session. 
The number of points that can be cap- 
tured varies based on the system used. 
However, typically you get from about 
eight to hundreds of sensor positions 
in either 2D or 3D. The data is com- 
monly brought into an animation sys- 
tem I ike Softimage or Maya and the 
data pointsdrivethedeformation of a 
model. Filmbox by Kaydara isdesigned 
specifically to aid in the process of cap- 
turing, cleaning up, and applying this 
form of data. Filmbox can also apply 
suppressive expressions, inverse kine- 
matic constraints, and perform audio 
analysis similar to Magpie Pro. 

This form of motion capture clearly 
can speed up the process of generating 
animation information. However, it's 
geared much more toward traditional 
animation and high -end performance 
animation. In this respect it doesn't 
really suit the real-time game develop- 
er's needs. It's possible to drive a real- 
time character by using the raw 
motion capture data to drive a facial 
deformation model. However, for a 
real-time game application, I do not 
bel i eve th i s i s cu rren tl y f easi bl e. 

In order to convert this stream of 
positional data into my limited real- 
time animation system, I would need to 
analyze the data and determine what 
vi semes and emotionsthe performer is 
trying to convey. You need a filtering 
method that will takethemultiplesam- 
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FIGURE 6. Magpie Pro simplifies the tasl( of isolating 
phoneme patterns in your audio tracl<. 



plepointsand selectthevisemeormus- 
cleaction that isoccurring. This is really 
the key to making motion capturedata 
usabi e for real-ti me ch aracter an i ma- 
ti on. This area of research, termed ges- 
ture recognition, is pretty active right 
now. There is a lot of information out 
therefor study. However, Quantum- 
works's Geppetto provides gesture 
recognition from motion capturedata 
to drive "muscle macros" as both a 
standaloneand a plug-in for Filmbox. 

Where Do Ue Go from Here? 

Between viseme-based and muscle- 
based facial animation, you can 
see th at th ere are a I ot of possi bl e 
approaches and creative areas to 
explore. In fact, the whole field has 
really opened up to game development 
in termsof opportunities for game pro- 
ductions as well as tool developers. 
Games are going to need contentto 
start filling up those new DVD drives 
and I thinkfacial animation isagreat 
way to takeour productionsto the 
next level. ■ 
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Talking Heads: Hierarchical Facial 
Animation in Real-Time 3D 




ntil very recently, facial animation was a technique reserved for full 
motion video and prerendered cinematics. The continuing advances in 
rendering technology and geometry-dedicated processors have opened 
the door for animators to use thistechnique within therealm of real- 



time 3D entertainment. 

Til is mon til's article is the first 
installment in a series dedicated to 
expanding the knowledge base of real- 
time facial animation. Overthenext 
few months, we'll discuss facial anato- 
my and skinning techniques, as well 
as phoneme recognition and anima- 
tion with linked expressions. We'll 
tackle the problem from the ground 
up, and by the end of the process we'll 



have covered all of the steps necessary 
to create and animate a speech-driven 
human head. 



Why Go to the Trouble? 

There is absolutely no logical rea- 
son why the actors in today's real- 
time 3D games shouldn't beableto 
smile, scowl, and talk with the player. 




FIGURE 1. Real-time head and face. 



Th e stru ggl e to create a bel i evabi e vi r- 
tual world is the struggle to create the 
illusion of reality. For any game based 
within a virtual environment, the 
player's enjoyment is directly linked 
to how immersive that environment 
feels. In the ideal case, players will for- 
get that they are sitting in front of a 
computer screen, and will lose them- 
selves for a few hours within thevirtu- 
al worlds we create. It is incumbent 
upon us as developers to use every 
means at our disposal to generate this 
effect. Facial animation can and will 
be one of the most effective tools for 
achieving this. When in-game charac- 
ters interact with the player through 
recognizable facial expressions and 
lip-synched spoken dialogue, we will 
have taken several steps towards 
achieving the perfect virtual world. 

Modeling the Head and Face 

The main reason we haven't been 
abl e to create bel i evabI e I i p- 
synched characters has been the ren- 
dering engines' polygon limitation. In 
a human head, theskin of thefaceis 
very plastic, in other words, extremely 
malleable. In order to achieve this 
effect convincingly and without 
excessive distortion, the polygonal 
densities of the face are required to be 
fairly high. Asa result of the recent 
improvements in rendering and pro- 
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cessing power, and through aggressive 
use of level-of-detail models, most cut- 
ting-edge engines will now accommo- 
date the relatively high density mesh- 
es required for facial animation. 



result, setting up a skeletal hierarchy 
at the beginning can still get you 
there because the mesh that's pro- 
duced for each skeletal animation can 
then be cloned and used as a morph 



The main reason we haven't been able to 
create believable lip-synched characters 
has been the rendering engines' polygon 
limitation. 



For our example we'll useahuman 
head, although the basic principles dis- 
cussed will apply to almost any face 
with the basic bilateral construction 
found in humans. The head in Figure 1 
has been modeled with sufficient poly- 
gons for facial expression, but still low 
enough forcurrent engine technology. 
Almost all of the head's 800 or so poly- 
gons have been devoted to the areas 
surrounding the eyes, nose, and 
mouth, where the bulk of facial expres- 
sion is displayed. In this case, we've 
actually goneto thetroubleto add the 
internalsof themouth, with tongue, 
teeth, and cheek surfaces included (this 
will be necessary if our character is 
going to bespeaking close up, in an in- 
game cut-scene, for example). 

Although this head has been mod- 
eled with real-life photographic refer- 
ence, which is often the best resource, 
Hogarth's Drawing the Human Head 
(Watson-Guptill, 1989), and Faigin's 
TheArtist'sCompleteGuideto Facial 
Expression (Watson-Guptill, 1990) are 
two excellent references with more 
generalized information. 

Why a Skeletal Hierarchy 
Instead of Morph Targets? 

Thejury isstill out on what isthe 
most efficient method for creating 
animated facial expressions. While 
generating morph targets can some- 
times be faster and often allows fine 
tuning of facial animation, taking the 
time to build a skeletal system with 
linked expressions can save time in 
the long run, especially when your 
game requires a large number of ani- 
mations. In any case, if you're sold on 
using morph targets as your end 



target. The bottom line isthat your 
technique will be determined by two 
factors: your engine's animation sys- 
tem, and your animator's expertise in 
the given method. 

Expressive Regions of the Face 

In order to set up our hierarchy cor- 
rectly, we need to identify the areas 
of the head and face that will be ani- 
mated. We get most of our informa- 
tion about a person's mood by looking 
at two distinct regions of the face. The 
highlighted areas in Figure 2 enclose 
the upper and lower "active regions" 
involved in facial expression. This is 



where we will invest the bulk of our 
time and energy setting up the hierar- 
chy. The upper facial node acts as the 
parent to the nodes controlling the 
eyes, eyelids, and eyebrows. Similarly, 
the lower facial node is the parent for 
the nodes of the lower jaw, mouth, 
and tongue. Breaking down the hierar- 
chy in this way will do two thingsfor 
us. First, by compartmentalizing the 
areas of facial expression, we have bro- 
ken down the problem of facial anima- 
tion into smaller, more easily managed 
tasks. Second, by creating a preset list 
of animation s for each region, we will 
be able to generatea widely varied set 
of facial expressions with relatively lit- 
tle effort. 



Bone Structure and Facial Muscles 

In orderto create the motion of the 
human face accurately, a basic 
understanding of the underlying mus- 
cle and bone structure is needed. Figure 
3 shows where each node corresponds 
to the mesh, while Figure 4 shows each 
node's relative position in the hierar- 
chy. As you can see, with a few excep- 
tions, the nodes are bilateral, and lend 
themselves to being moved in pairs 
1* Theskull. This will serve as the 
top node for thehierarchy. Several of 




cpressivt;] Regions 
of thf Face 



CorTfispnndin^ Hierarchy 



FIGURE 2. Regions of the face and nodal hierarchy. 



http://www. gdmag.com 



juLYiggg game developer 



ARTIST*S VI EW 





FIGURE 3. Nodal placement. 



the muscle groups involved in facial 
animation are anchored to this bone. 
Although this node does not animate 
and could be represented by any arbi- 
trary shape, It Is useful to approximate 
the general shape of the skull when 
creati n g a h I erarch y, as th i s serves as 
the foundation for and aids In the 
placement of subsequent nodes. 
2* Upper eyelid node (levator palpe- 



lid is kept open largely by the force of 
gravity. This node works in conjunc- 
tion with number 9 below. 

5* Tongue NODES. The tongue Is an 
optional part of the hierarchy, 
although if you want to use close-up 
shots of the characters. It Is a definite 
necessity. The muscles of thetongue 
are particularly versatile, and are 
among the strongest In the body. For 



For any game based within a virtual 
environment, tlie player's enjoyment is 
directly linlied to how immersive that 
environment feels. 



B/Mf), This raises the upper eyelid, as in 
a surprise or fear response. Because the 
upper eyelid hasltsown musclegroup 
and the lower does not, most of the 
motion associated with the eyelids 
occurs In the upper eyelid. 

3* Eye node. The mesh of the eyeball 
should be separated from the rest of 
the face so that It can turn freely with 
the node. Also consider placing a con- 
straining expression on the eye nodes, 
so th at th ey move i n con cert. 

4* Lower EYELID NODE. The lower eye- 



th is reason, the full flexibility of this 
muscleshould be represented by no 
fewer than three bones or nodes. 

6* The JAWBONE. Pay particular atten- 
tion to the pivot point for this node. 
Thejawbone Isa hinged joint, with the 
pivot point located at the extreme rear 
point on the bone, where It Is hinged 
to theskull. The tongue nodes and 
most of the lower mouth nodes are 
children of this node. 

7. The FRONTALIS. This muscle is 
responsible for raising the eyebrows 



vertically. A bilateral sheet-like mus- 
cle, it is connected to the thick fiber of 
the scalp Immediately beneath the 
hairline, and Inserts Into the skin 
directly under the eyebrows. The 
action of the frontalis Is seen In such 
expressions as surprise, sadness, and 
fear, although it also sees action dur- 
ing regular conversation since raising 
the eyebrow Is one of the most com- 
mon facial gestures. We often do this 
in concert with or In place of hand 
gestures during normal speech. 
Although this muscle can be repre- 
sented by a single node, many people 
have control over the Individual sec- 
tions of the frontal Is, enabling them 
to raise one eyebrow, while lowering 
the other (the"Spock" eyebrow). For 
thisreason, there are two nodes, one 
for each side of the face. 

8* The coRRUGATOR. This muscle 
group, also known as "the scowling 
muscle," Is actually comprised of two 
muscles, the corrugator and the pro- 
cerus. These muscles anchor to the 
skull at thetop of thenasal cavity, and 
at the Inside corners of either eye sock- 
et. The basic function of the corrugator 
Isto pull the eyebrows down , whileat 
the same time bringing them closer 
together. Used primarily in conjunc- 
tion with thefrontalls, this muscle is 
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FIGURE 4. Each node's relative position in the hierarchy. 



of the upper lip, which isanchored at 
each end by the muscles at the corners 
of themouth. Also called the"lip 
tightener," this muscle works in con- 
junction with the levator labii superioris 
to create the expressions of disdain 
and loathing. It isalso used for pursing 
the lips, as when someone experiences 
deep thought or concentration. 

12* ZYGOMATICUS MAJOR, RISORIUS/ 
PLATYSUA, AND THE TRIANGULARIS. These 

th ree musci e groups are respon si bl e 
for pulling the corners of the mouth 
up, out, and down, respectively. 
Approximated hereby the action of a 
single node, these muscle groups are 
active over a range of expressions, 
from the exclamation of joy and plea- 
sure, to the extreme stress of pain or a 
tragic loss. To effectively mimic the 
action of these muscles, it is necessary 
to incorporate most of the nodes in 
the lower region of the face. Test this 
out by smiling broadly or frowning 
severely, and you will seejust how 
much real estate this muscleaffects. 
Thiswill also be one of the most 
active nodes, since for any speech or 
mouth movement, this node will be 
used to approximate the motion at 
the corners of the mouth . 

13* Depressor labii inferioris and the 
MfMMi/s. The combined action of these 
two muscles tends to pull thelowerlip 
down (as during speech), or to push it 
upwards (giving a pouting expression). 
Here again, theopposing motion of 
two separate muscle groups can be 
approximated by a single node. 



associated with the acts of crying, 
scowling, and extreme concentration. 

9 • Orbicularis oculi. Thisisalarge 
muscle group made up of concentric 
rings of muscletissue, totally encircling 
the eye and extending into the cheek. 
Asthismusclecontracts, ittendsto 
squeeze the eyes shut whileat thesame 
raising a good portion of the skin of the 
cheek. This muscle sees action whenev- 
er we squint, laugh, or smile, and isthe 
primary operator in expressing pain. 
Although this muscle group can be 
approximated by two separate nodes (as 
in Figures), a singlespherical node 
en com passing the eye can also be used. 
In that case, scaling the node down in 
thex, y, and z axiswould approximate 
the contracting of the muscle group. 



10* Levator labii superioris. Also 
called the "sneering muscle," thismus- 
clegroup usesa three-point anchorage 
spanning from the bottom edge of the 
eye socket to the lower ridge of the 
cheekbone. The muscles converge to a 
single point and insert into the skin 
just above the upper lip. Contracting 
thismuscletendsto raise the upper lip 
towards the nostrils. In thereal world, 
this muscle seldom sees action, except 
when expressing disgust, disdain, or 
loathing. We approximate it hereby 
using a singlenode, which for us will 
be additionally useful in properly shap- 
ing the skin of the cheeks over several 
different expression groups 

11 • Orbicularis oris. Th i s i s a ban ded 
group of muscles just under the surface 



Wrap Up 



Now that we've identified the major 
facial muscle groups and created our 
hierarchy, it'stimeto apply theskeletal 
structure to themesh, and weightthe 
facial vertices appropriately. The last 
step will be to set up a lip-synching 
table and expression list, and from 
there we'll be able to start animating. 
That's where we'll pick up next month. 

For more information about facial 
animation from a programmer's per- 
spective, please review Jeff Lander's 
columns, "Read My Lips: Facial 
Animation Techniques" (Graphic 
Content, June 1999) and "Flex Your 
Facial Animation Muscles," which 
appearsin thismonth'sissueof Game 
Developer. ■ 
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Technology Update: Dominotrix for Softimage 



In May's column, "See Jane Walk," we examined character 
animation tecliniques used in the industry, focusing partic- 
ular attention on motion capture. Until very recently, 
motion capture animation in Softimage was a painfully rig- 
orous exercise that limited animators to using a skeletal hierar- 
chy based on a nodal constraint system. In layman's terms, the 
skeleton which the motion capture data fit onto was not neces- 
sarily the same type of skeleton which animators preferred to 
use while animating. The mo-cap skeleton had to match the data 
exactly, or the animation wouldn't work. Subsequent data 
manipulation on a mo-cap skeleton often proved restrictive, so 
that mixing modes of classical animation with mo-cap was prob- 
lematic at best. Thanks to the team at House Of Moves, mo-cap in 
Softimage just got a whole lot easier. 

Taylor Wilson, the CTO of the Los Angeles-based motion cap- 
ture studio, has spearheaded a technique dubbed Dominatrix, 
which works to separate the motion data from the skeletal hier- 
archy; in essence, this technique frees up the animator to use 
whatever style of skeleton with which he or she feels most com- 
fortable. For example, say you want to use motion capture data 
for your character, but you've already started animating using 
another method. With the previous restrictions in Softimage (and 
most other animation tools), you would have to scrap all your 
previous work in favor of the mo-cap skeleton which matched the 
mo-cap data. Now, with Dominatrix, you can combine previously 
recorded motion capture data with other forms of character ani- 
mation without ever having to change your skeleton. 



How Does It Work? 

The process is extremely simple. You merely provide 
House Of Moves with your character's skeleton (sub- 
mitted in a Softimage .HRC file), and they do the rest. 
The motion capture data, originally stored in the 
Acclaim format, is washed through a set of proprietary tools 
which correct for any differences in proportion, orientation, and 
number of nodes between the standard Acclaim skeleton and your 
character's skeleton. Then the data is mapped onto your charac- 
ter's skeleton, and saved out as a Softimage .ANI file. Dominatrix 
can also generate IK constraint information to drive any of the 
limbs of your character. (This is in place of or in addition to using 
tradition positional/rotational keys at each joint.) The level of 
customization possible allows you to set up your character pretty 
much any way you want, with the simple restriction that the char- 
acter's skeletal hierarchy has the same parenting relationships as 
the mo-cap skeleton. This is illustrated in Figure i, which shows 
several humanoid skeletons all sharing the same hierarchical 
relationship (two arms, two legs, a torso, and so on). Note that 
the limbs of each skeleton are different in length (some have long 
arms, some short) and that the number of nodes in any given limb 
can differ between characters (the War Giant character, for exam- 
ple, has four nodes in his leg whereas the human character has 
only three). Dominatrix can accommodate all of these skeletons 
with the same set of motion capture data. 



Since this is something I had to see for myself, I opted for a 
test run of Dominatrix using Drakan's main character, Rynn. Her 
Softimage skeleton is chock full of extra nodes in her chest, 
arms, and legs, and doesn't match up very closely to the Acclaim 
skeleton. Within a few days of sending off the Softimage skele- 
ton to House of Moves, I received several martial arts animation 
files (see Figure 2) which had been mapped onto the character. 
When I imported these onto Rynn's skeleton they worked flaw- 
lessly, and I have to say it was the least painful experience with 
motion capture I've ever had. The technique is called Dominatrix, 
and it's available (at present, exclusively) from House Of Moves 
(http://www.moves.com). —Mel Guymon 
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FIGURE 2. Drakan's main character performing a martial 
arts move. 
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Mpath, HearMe and Mplayer 



remember talking to an Electronic Arts executive at Intel's Pentium III launch 
this past March. He was both roused by his company's plans in the online 
gaming market, and very self-conscious. I wondered if he was nervous about 
the prospects, or whether he was concerned that no oneelse would jump in 



and takea big biteof thepie. In April, 
wlien M patli decided to go public, I 
liad aclianceto think some more 
about til e issues facing tin e server side 
of tlie game business. There are some 
sobering lessons to be learned from 
Mpath, not only about onlinegaming, 
but also about what it takes — beyond 
games — to make it in cyberspace. If 
you look closely at M path's strategy 
you'll find that it could almost form 
thefoundation of a big publisher, such 
asElectronic Arts, going wider than its 
core demographic, perhaps into the 



realms occupied by thereally bigfish, 
such as Disney Online. 



The Online Gaming Business Model 

path was founded in January 
1995. The company states as its 
business practice that it operates and 
licenses live community Internet sites 
to such companiesasSegaSoft 
Networks. Mplayer.com isthecompa- 
ny's low-latency gaming platform, or 
live community as it is called, that 



FIGURE 1 . Mpath's statement of operations data for the fiscal years indicated as 
a percentage of total revenues, n/m means "not material." 






1996 


1997 


1998 








Net Revenues: Live Communities 


9-7% 


25.2% 


37.6% 


Foundation 


90.3 


74-8 


62.4 


Total revenues 


100 


100 


100 


Cost of net revenues: 








Live Communities 


28.2 


66.3 


27-7 


Foundation 


72.6 


22.7 


9.8 


Total cost of revenues 


100.8 


89 


37-5 


Gross profit (loss) 


(0.8) 


11 


62.5 


Operating expenses: 








Research and development 


n/m 


89.3 


39 


Sales and marketing 


n/m 


253.2 


97.8 


General and administrative 


n/m 


104.2 


40.8 


Stock compensation 


n/m 


61.5 


32.4 


Write-off of acquired intangibles 


n/m 






Total operating expenses 


n/m 


508.2 


210 



Omid Rahmat is the proprietor of Doodah Marketing, a digital media consulting firm. 
Healso publishes research and market analysis notes on his web siteat 
http://www.smokezine.com. 



they deployed with the help of PSINet 
in late 1995. In January 1999 thecom- 
pany launched HearMe.com, its sec- 
ond live community site. Whilemost 
game developers are familiar with 
M player.com, they may not know as 
much about HearMe.com. 
HearM e.com currently consists of 
seven live audio communities, making 
live audio interaction availableto peo- 
ple whose interests extend beyond 
entertainment. Mpath managed to 
raise approximately $34.9 million 
between its inception and the end of 

1998 through the sale of equity securi- 
ties to CSK, SegaSoft's parent compa- 
ny, and various venture capital (VC) 
and strategic partners. In January 

1999 the company raised an addition- 
al $20 million through VCsto prepare 
for going public. That's a big chunk of 
change, but Mpath hoped to raise over 
$68 million from its initial public 
offering (IPG). 

Being an onlinecontender requiresa 
big investment, even in thegaming 
nichethat Mpath has carved for itself, 
but their long term strategy seems to 
indicatemuch more. The company 
derives its revenues from two business 
units. Live Communities and Mpath 
Foundation. LiveCommunities generate 
advertising revenues, and this includes 
adverti semen ts targeti n g th e key demo- 
graphicsat Mplayer.com and 
HearMe.com. Mplayer.com consumers 
are predominantly male. In sports and 
game player communities, more than 
90 percen t of th e parti ci pan ts are mal e, 
and the people within those communi- 
ties are typically between the ages of 13 
and 50. However, approximately 40 per- 
cent of the participants in classic games 
and the casino community are women. 
Thepeoplewithin these communities 
are typically between theagesof 25 and 
50. As for HearMe.com, currently more 
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than 40 percent of the participants are 
women, and its members are typically 
between the ages of 13 and 55. 

Mpath Foundation isthe licensing 
and services arm of the company. In 
1998, CSK Sega, Sony and Electronic 
Arts accounted for 23, 12 and 10 per- 
cent of total revenues respectively for 
Mpath Foundation. In 1997, CSK Sega 
and Sony accounted for 35 and 11 per- 
cent of total revenues respectively. 
Combineboth LiveCommunitiesand 
M path Foundation, and the business 
model for Mpath is simple. The compa- 
ny created Mplayer.com, and now 
HearMe.com, to showcase theapplica- 
tion of its technology, and to put up a 
barrier to market entry for other parties 
interested in competing with so-called 
live communities. There's a growing 
mesh of third parties that tie in with 
M path's live communities, whether 
they be advertisers, vendors who have 
products that sell directly through 
M path operations, or magazines and 
portalsthat linkto Mpath'sdemo- 
graphics Mpath Foundation then takes 
the technologies that make all this 
happen, and applies them to other 



placeson theweb. POP.X istheprod- 
uctthat Mpath Foundation sells, a 
toolkit for enabling live communities. 
And here is where it gets interesting, 
and where the lessons emerge. 

Mpath is, like most Internet compa- 
nies, primarily interested in growth. 
This means pushing up memberships 
and subscriptions. In order to do that, 
thecompany needscontent from third 
parties, and that makes for more enter- 
taining reasonsto build an audience. 
Furthermore, Mpath isalso putting a 
lot of money into research and devel- 
opment of key technologies that will 
better manage its communities, and 
streamline the low-latency web enter- 
tainment experience. 



Growth Maxims 

path Foundation consists of a 
growing list of on line entertain- 
ment companies, including CSK Sega, 
Electronic Arts, Fujitsu, GTECH, and 
LG Internet. TheMplayer.com service 
now comprises three active communi- 
ties built around common interests. 



and offers more than lOOofthemost 
popular online multiplayer games. 
According to M path's internal compa- 
ny reports, total usage time on 
Mplayer.com exceeds 200 million user- 
minutes per month asof January 1999, 
compared to 67 million user-minutes 
per month back in January 1998. 
Mplayer.com has becomethetenth 
largest Internet site in terms of total 
usage time per month, according to the 
company's own usage data. 

The opportunities for Mpath are 
extensive. The technology, thefact 
that they are growing beyond online 
gaming (a feat that some of their more 
esoteric competitors have failed to 
negotiate), and the general branding of 
the company's products all have 
helped thecompany achievea leader- 
ship role. However, with almost two- 
third s of th ei r rev en ue com i n g from 
th ei r tech n ol ogy I i cen si n g an d servi ces, 
and advertising and subscriptions 
being very speculative means of mak- 
ing a profit on the Internet, Mpath 
faces some chal lenges. 

First of all, there isthe Ultima 
On LINE franchise model. It's worked for 
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Electronic Arts, and it means that pub- 
lishers are willing to risk someeffort 
and cash in order to create their own 
live communities. Being game develop- 



FIGURE 2 . Statement of operations (in tliousands of 
dollars) from January p, 1995, to year ending 
December 31, 1998. 



I 



Foundation 




112 


2,041 


Total revenue 




124 


2,727 


Cost of net revenues: 








Live Communities 




35 


1,808 


Foundation 


- 


90 


620 


Total cost of revenues 




125 


2,428 


Gross profit (loss) 




(1) 


299 


Operating expenses: 








Research and development 


1,502 


5.261 


2,436 


Sales and marketing 


171 


3.937 


6,906 


General and administrative 


746 


2,877 


2,841 


Stocl< based compensation 


34 


383 


1.676 


Write-off of acquired intangibles 




12,876 




Total operating expenses. 


2.453 


25.334 


13.859 


Loss from operations 


(2.453) 


(25.335) 


(13,560) 


Interest and other income 








(expense), net 


99 


291 


(93) 


Loss before provision 








for income taxes 


(2.354) 


(25.044) 


(13.653) 


Provision for income taxes 


(1) 


(1) 


(1) 



ersat heart means that these publishers 
will probably want to build the server 
technologies themselves, and tailor 
them to specific gaming experiences 

and genres. In other 

words, the one-size-fits- 
all approach may not be 
the ideal way to go for- 
ward. Of course, M path 
also makes a good acqui- 
sition target for a com- 
pany that wants to get a 
head start, and despite a 
very successful IPO, the 
volatility of Internet 
stocks could have an 
impact on the future 
growth of the company. 

The real lesson to be 
learned from Mpath is 
that in order to succeed 
in theonline world you 
haveto juggle both your 
investment in sales and 
marketing to draw traffic 
and your spending on 
research and develop- 
ment to put up technolo- 
gy barriers to competi- 



tion. These technology barriers are 
becoming increasingly difficult to main- 
tain, relying as they do on similar objec- 
tives having to do with managing and 
maintaining onlinecommunities. 
Everyonewants to trap theconsumer 
on their site and keep them there. As for 
spending lots of money to get traffic, 
that depends on content. Otherwise, 
your customer has to do very little in 
order to go somewhere else. Point, click, 
and carriage return isaboutall it takes. 
And that's the best news that smaller 
developers have heard in a long time. 
Building communities around content, 
developing a franchise, and creating 
multiple revenue streams are all within 
the reach of any Internet savvy develop- 
ment group. As higher production costs 
raise the barriers to entry in the retail 
game market , and a handful of publish- 
ers control distribution , theonline 
world offers the biggest meansof access 
to thegameenthusiast. Thetrick is 
keeping the consumer from turning the 
page, but isn't that the problem for 
every creative venture? Keep th e audi- 
ence coming back for more. Now go fig- 
ure out how. ■ 
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Using the Je\Ve^ Native 

interfe^ce \^itbin Oe\>^es 

BY BERND KREIMEIER 

ou have heard of Java. Actually, you probably have had a hard time trying to escape the 
hype surrounding it since 1995. There are, however, compelling reasonsto usejava in 

shrink-wrapped games. Two major optionsthat have recently been 
examined by professional game developers are using Java as your 
scripting language, and usingjava for safe run-time downloadablecode 

m 

that gets executed on theclient. In both cases, it is the interface to native 
code that you end up dealing with thesamegluethat is required to makejava 
work on your PC in the first place. If using a portable, standard, object-oriented 
programming language with built-in security within your game sounds appealing, 




then you should become acquainted with thejava Native Interface (J N I), which isyour 
tool to write "dirty" Java — Java code that istightly integrated with your native code. 



Bernd Kreimeier is a physicist, a writer of novels and articles, and a coder. Currently living in Ireland, he is doing con- 
tract work on Java for games, and working on W arped Space, his own game design. This gun for hire— contact 
bk@gamers.org. 
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There is an abundance of information 
about "pure" Java, and this is not the 
place to explain all theactual and 
al I eged ad van tages of J ava tech n ol ogy . 
Unfortunately, many of the highly 
touted "pure" solutions quietly omit 
the sophisticated native machinery at 
work under the hood. Instead of look- 
ing at gameapplets running in web 
browsers, this article sizes up possible 
real-world usesforjava, and looks at 
the ways some game developers are 
already usingjavafortheirtitles. 



A Brief Recap of Virtual Machines 

Briefly, here are the key compo- 
nents th at are rd evan t to th i s d i s- 
cussion of Java: 

Java bytecode. This is pseudocodefor a 
stack-based processor described in the 
Java specification. Valid bytecode has to 
satisfy a lot of requirements, but if you 
really wanted to, you could actually 
write it by hand with a decent hex edi- 
tor and the specifications. 
The Java Virtual Machine (JVM). Java CPUs 
have not yet conquered the market, so 
software mu st tran si ate J ava i n structi on s 
into the language the PC hardware can 
understand. The JVM isa multithreaded 
program for your operating system, sup- 
plied from various vendors, that exe- 
cutesjava bytecode and maps bytecode 
to native instructions. Thejava specifi- 
cation pi aces few restrictions on the 
actual implementation of a JVM . You 
can find Open SourceJVMson the 
Internet, or even write your own clean- 
room implementation. You can also get 
the sources of Sun's referencejava 
Development Kit (JDK) and negotiate a 
license for commercial use. 
The Java Programming Language. This is an 
object-oriented language, designed 
with a subset of C-H-in mind. It has 
run-time bounds checking, it is restrict- 
ed in terms of memory access and han- 
dling, and it has simplified inheritance 
patterns. The language features an 
overwhelming inflation of extension 
APIs and core classes. 

It is important to understand that 
these three components are entirely 
separate. AJVM will happily execute 
valid bytecode that was not generated 
from Java source — for instance, from a 
compiler that maps a language like 
Scheme to bytecode. You could also 
implement a simplified virtual machine 



that skips validation and executes byte- 
code no compliant JVM would accept, 
or create a virtual machinethat hasno 
garbage col lector thread at all. In addi- 
tion, there are compilers that generate 
native code from Java source files — 
you don't need a JVM to write an appli- 
cation in Java. From the "amusing 



FIGURE 1 . JNI built-in data types. 



FIGURE 2 . JNI visible class tree. 



avenue of hand-tuned virtual assem- 
bly" (as John Carmack of id Software 
referred to virtual CPUs) to using aJVM 
without ever coding in Java (converters 
mapping C or C-H- subsets to J ava are 
feasible), all your tampering needs can 
somehow be addressed. 
A lot of confusion originates from dis- 





Java 


JNI Name 


Values/Size 


JNI Signature 


boolean 

byte 

short 

int 

long 


jboolean 

jbyte 

jshort 

jint 

jlong 


JNLTRUE / JNLFALSE 
signed 8 bits 


z 

B 
S 
1 
J 


signed 
signed 


32 bits 
64 bits 


float 


jfloat 


IEEE 754 


32 bits 


F 


double 


jdouble 


IEEE 754 


64 bits 


D 




char 


ichar 


Unicode* 


16 bits 


C 




void 


void 


N/A 




V 



* The char data type Is the only unsigned integral type available in Java. It is interpreted 
as i6bit Unicode, and mapped from/to UTF-8 encoding in I/O translations, but it is 
expanded to int for arithmetic operations, and all integral arithmetic operations 
are available. 



Java Class name 



java.lang.Object 



JNI handle 

jobject 



JNI signature 

Ljava/lang/Object 



java.lang.Class 

java.lang.String 

java.lang.Throwable 

N/A* 



jclass 
jstring 
jthrowable 
jarray 



Ljava/lang/Class 
Ljava/lang/String 
Ljava/lang/Throwable 
N/A* 



java.lang.Object[] 
L java.lang.Stringj] 



r, 



boolean!] 
charU 



ML 



shortf] 



ML 



longf] 



floatj] 



doubled 



jobjectArray 
jobjectArray 

jbooleanArray 

jcharArray 

jbyteArray 

jshortArray 

jintArray 

jlongArray 

jfloatArray 

jdoubleArray 



[Ljava/lang/Object 
[Ljava/lang/String t 

[Z 

[C 

[B 

[S 

[i 

[L 

[F 

[D 



* There is no java.lang.Array class, so JNI's jarray has no real equivalent in Java, just like jmethodlD, 
jfieldID or jvalue. 

t There is no jstringArray, so arrays of Strings map to the jobjectArray handle. So do arrays of Class, 
Throwable and any other subclass of Obiect, as well as mixed arrays and arrays of arrays. 
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cussionsthatfail to distinguish thevarl- 
ouscomponentsof Java technology. All 
of these are required parts of any JDK, 
however. TheonlyJDK that Isactually 
called JDK Isthe reference Implementa- 
tion provided by Sun Microsystemsfor 
Solarlsand Win32, and itslicensed 
ports to other platforms. 

The concept of a virtual machinefor 
game scripting is not new, and not 
restricted to Java. John Carmack 
recently decided to use a custom virtu- 
al machineand bytecode created by a 
modified C compiler (LCC by Fraser 
and Hanson) for id's upcoming game. 
Quake 3: Arena. Carmack once pointed 
out that game coders "have more 
urgent things to do than design lan- 
guages." Ironically, he is now engaged 
in designing hisown virtual machine 
and native interface. Technologies like 
Java'sjust-in-timecompilation and 
HotSpot optimization originate in the 
Java technology mainstream, and they 
are powered by more resources than a 
single game company could ever com- 
mand. If you can make Java work for 



your game, then you will benefit from 
this momentum. 



Talking to the Natives 

Came coders usually do not trust 
cross-platform APIs based on lay- 
ers of abstraction. Interpreted bytecode 
does not typically appeal to an indus- 
try that still counts on assembly to get 
a performance edge. Windows-based 
games sell, period, and portability is 
not really an issue. 

Compare thisto theholy grail of "100 
percent purejava." Mainstream Java 
technology isseemingly meantfortiny 
"gamelets," not serious games. Besides, 
despite the bloat, there are bits and 
pieces missing from thejava core class- 
es: access to certain devices and system 
services issimply not available (and 
might never be, for design and security 
reasons), and politics sometimes gets in 
the way (witness the lack of Java 
OpenGL bindings). However, if you take 
a closer look, it turns out that there is 



LISTING 1. Native code making use of Java. 



II Java class, server-side Game Logic scripting, 
package somegame; 
class ScriptEngine { 

// This uses native errorO callback. 

public static void initO {..} 

public static native void error( int code ); 

} 

♦include <Jni.h> 

// Native error callback provided to Java, 
extern void SI/_ErrorScriptEngine( jint i ); 

JNINativeHethod svSE = {"Java_somegame_ScriptEngine_error","(I)V", SV_ErrorScriptEngine }; 
// Native function to set up scripting module, 
void SV_InitScriptEngine( JNIEnv* env ) { 

j class clazz = NULL; 

jmethodID method = NULL; 

jint err; 

// Lookup class, loads the class if not yet done. 

clazz = (*env)->FindClass( env, "somegame. ScriptEngine" ); 

if ( clazz != NULL ) { 

// Register native method, returns zero on success. 

err = RegisterNatives( env, clazz, isvSE, 1 ); 

// Lookup Java methodlD. 

method = (*env)->GetStaticMethodID(env, clazz, "init", "OV"); 

} 

// Handle errors. 

// Call static method. 

(*en V ) ->CallStaticVoidHethod (en v , clazz , method) ; 



alwaysnativecodeatthevery heart of 
all that "purejava": thereisaJVM writ- 
ten in native code, and core classes part- 
ly implemented as native code. Here 
reignsthejava Native Interface, gluing 
together Java and nativeC/C-H-code, 
and it isthekey to combining your 
n ati ve code with J ava tech n ol ogy . 

JNI is part of thejava specification, 
and it's a mandatory part of all Java 
implementations. Sun was recently 
granted a court injunction forcing 
M icrosoft to add JN I to the M icrosoft 
Java implementation. Ideally, .DLLs 
and binaries usingJNI should be byte- 
compatible for a given platform. The 
1997JNI specification isavailable 
online, and there are also books on the 
subject, sothisarticleincludesonly a 
brief summary of it before we get into 
its applicability in games. 

Thefirst task JNI must solve is getting 
theJVM and user-written native code to 
agree on built-in types and memory lay- 
out to exchange data (see Figure 1). 
Only some of the core classes are repre- 
sented forthenativecode(see Figure2) 
— most of them arrays of thebuilt-in 
types. 

What about jclass and jobject?JNI will 
not hand you the memory layout of a 
Java object, but it must provide you a 
handle. It even preserves the relation- 
ship between Java. lang. Object and 
java.lang.Qass. A jclass object can be cast 
to jobject safely in anyJNI that complies 
to thejava specification (non-compliant 
implementations have been found). JNI 
isforemost aimed at C (theC-H-bind- 
ingsarejust inlined wrappers), and no 
support for object-oriented program- 
ming on the native side is offered. With 
the exception of Throwable, String, and 
arrays, all classes have to be squeezed 
through the jobject and jclass represen- 
tation. Arrays of arbitrary classes 
(including String) will always be mapped 
to jobjectArray. JNI defines JNI.FALSE/TRUE, 
a jvalue union type, and a jsize foryour 
convenience. 

Further, handleshave access to fields, 
and also call methods of classes and 
objects. It might look like java. lang. - 
reflect. Field and java. lang. reflect. Method 
are thejava equivalents toJNI'sjfieldlD 
and jmethodID, but JNI predated the 
Reflection API, and actual reflection 
support was added only to thelatestJNI 
revision. 

Caching field and method IDs is a 
good idea, as retrieval involves a string 
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lookup. Be warned that caching can get 
tricky in applications with multiple 
threads and class loaders. You will have 
to keep an eye on the garbage col lector 
as well — without a strong reference 
acquired by NeuGlobalRefO, the garbage 
collector might remove the object your 
native code is still referring to. 
Likewise, dangling references not 
removed by DeleteGlobalRef () can keep 
obsoletejava objects from being col- 



lected. UseDeleteLocalRefO to avoid 
accumulating temporary references 
within loops JDK 1.2 offers limited 
support for weak references, too. 

Within your native code, all will 
revolve around theJNlEnv interface 
function table — your door to thejava 
side. It provides methods to handle ref- 
erences, create objects, load classes, 
access fields and call methods. Further, 
you get utility functionsto iterate 



arrays, throw exceptions, and perform 
monitoring to makethe native code 
threadsafe. Finally, an executable can 
also register functions as native meth- 
ods, making code known to theJVM 
without dynamic linkage. 

Listing 1 is a small example showing 
how native code can use Java to get 
thingsdone, and how a native call back 
is registered with theJVM . Listing 3, 
discussed later on, does the opposite: it 
sh o ws h o w J ava cal I s n ati ve cod e. 

It is tempting to use static (class) 
methods, as you do not have to handle 
an object in addition to the class. In 
many cases this is absol utely sufficient 
— servers will likely not run two script 
enginesin parallel. In many other cases 
though, thisleadsto bad object-orient- 
ed design on thejava side. 



Double Indirection: The Catch-22 

There have been competing native 
interface APIs proposed, most 
notably M icrosoft's Raw Native 
I nterface{RN I). The problem with RNI is 
that it exposes the underlying operating 
system andJVM implementation, 
which makesit impossibleto port to 
another VM . 

In some ways, theproblem with JNI is 
that it does not expose theVM imple- 
mentation. JNI makes you go through 
painsto en sure that native code never 
gets to seehowjava objects arelaid out 
in memory. Consequently, the native 
code has to deal with indirections every 
step along the way, many of them ulti- 
mately leading to tableand string 
lookups. This is not good for application 
performance. Seetheexamplein Listing 
2, where we pass command lineargu- 
mentsfrom Javato C, which involves 
references, arrays, and conversion to 
UTF-8 (the canonical two-byte Unicode 
encoding used byjava). 

Tools such asjauah generate C header 
files containing proper function proto- 
types (name and signatures) from a J ava 
class. These tools are already Unicode- 
aware, thus using underscores and 
other special characters in Java method 
and class names can lead to surprising 
results. The code in GAME. DLL will be 
linked to the class by theJVM by call- 
in g j ava .lang . System .loadLibrary ( "Game" ) 
automatically. 

The example in Listing 2 implements 
a minimal Java wrapper around native 



LISTING 2 . Passing Arguments from Java to C. 



II Minimal Java control code wraps legacy engine, 
package somegame; 
class GameMain { 

protected static native int nativeHain( StringD args ); 

public static void main( String[] args ) { 
int ret = nativeHain( args ); 

} 

static { System. loadLibrary("Game") ; } 
} 

// Handing over the arguments to native code. 
// This code will be put into the Game. DLL. 
♦include <jni.h> 

extern int gameHain( int argc, char** argv ); 
JNIEXPORT jint JNICALL 

Java_somegame_GameHain_nativeHain ( JNIEnv* env, jclass els, jobjectArray jargv ) { 
jint res; 
jint argc; 
jint i; 

jboolean isCopy; 
jstring jstr; 
jsize len; 
const char* cstr; 

jargc = env->GetArrayLength(jargv) ; 
for ( i = 0; Kargc; i++ ) { 

jstr = (jstring)(*env)->GetObjectArrayElement(env,jargv,i); 

cstr = (const char*)((*env)->GetStringUTFChars( env, jstr, &isCopy )); 

// We copy - we have to release, and we don't want to accumulate local references. 

argv[i] = (char*)malloc( strlen(cstr)+l ); 

strcpy( argv[i], (const char*)cstr ); 

// Did the JVM copy as well? 

if ( isCopy == JNI.TRUE ) { 
(*env)->ReleaseStringUTFChars( env, jstr, cstr ); 

} 

// Clear local reference. 
(*env)->DeleteLocalRef (env, jstr) ; 

} 

// Call our mainO now. 
res = (jint)gameHain( (int)argc, argv ); 
// Release allocated memory, 
for ( i=0; Kargc; i++ ) { 
free( argv[i] ); 

} 

// Return to Java, 
return res; 
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legacy code. Given all the implicit and 
explicit copying, we somehow seem to 
have come full circle: to get rid off some 
portability-related Java overhead, we 
decided to use native code, only to find 
outthattheJNI design hampersthe 
interaction between Java and native 
code to ensure portability. Now what? 
W el I , th ere are basi cal I y two ways I eft to 
increase performance: 

1* Brute force. You could switch 
toolsand compileto native code. If 
you pursuethis option, make sure 
yourjava compiler supportsJN I as 
well, and that it doesn't just compile 
pure bytecode. 

2* Smart design. You could accept 
the limitations of theJNI, and design 
your native and Java modules in a 
way that streamlines the interface 
between them. 

Mind you, your native code by itself 



will be as fast as it gets It'sonly the 
transfer of parameters and results back 
and forth that, insidean inner loop, in- 
curs significant performance penalties. 



The Invocation API 

Now that you have seen some 
meansto gluejava and native 
code together, where does a game 
developer actually get access to the 
virtual machine? The common answer 
is the Invocation API. The Invocation 
API allows you to embed theJVM into 
your native applications. It provides 
the means by which you can retrieve 
an existing JVM attached to your 
application, or launch one with prop- 
er configuration settings. Listings 
shows how to invoke theJVM in an 
application, usingJDK 1.2. You can 



use code like that shown in Listing 1 
to get Java classes loaded and execut- 
ed. If you do not want to encapsulate 
native method code into .DLLs, then 
your application can usetheJNI func- 
tion RegisterNatii/esO to make native 
functions from the executable known 
to theJVM . That way your game 
would ship without any .DLL. 

Most JVM and compiler implemen - 
tati on s fully support JNI, as it is need- 
ed to handle cleanly implemented 
core classes. (Interestingly, even Sun's 
own JDK doesnot always useJNI 
internally.) But the Invocation API 
was sometimes omitted from JDK 
ports and third-party J VMs. If you 
want to use the Invocation API, make 
sure your tools and targets support it. 

Worse still, someJDK ports support 
Invocation, but do not do so properly 
(including some revisions of Sun's own 
Solaris reference implementation). 
Invocation requiresthreadsafe.DLL 
handling, which isnot always granted 
(for instance, in some Solaris and Linux 
revisions). If thedynamic linking isnot 
threadsafe, your application will suffer 
spurious errors during startup. 
Furthermore, the official Java specifica- 
tion now sanctions limitations of the 
referenceJDK that affects DestroyJavaVHO. 
It isnot possibleto destroy aJDKJVM . 
Consequently, you can't invoke anoth- 
er onefrom the same application — 
multipleJVMs, whether subsequent or 
in parallel, are not possible. Once you 
loseyourJVM for whatever reason, 
your application must terminate. Fortu- 
nately, most of the other pitfalls were 
smoothed out last year. 

In Listing 3, ignore theJVM handle, 
which you can always retrieve by call- 
ingtheJNI function GetJavaVNO. A more 
flexible approach to invoke theJVM 
callsGetCreatedJavaVHsO first to check 
whether a JVM already exists, and uses 
AttachCurrentThreadO to make itself 
known if one isfound. Thequestion 
is, do you really have to invoke the 
JVM this way? 



Two Architectures: Embedded Java 
and Encapsulated Native Code 

How you choose to obtain theJVM 
for your game is, in all likelihood, 
the most important decision you have 
to make when rigging up ajava-based 
game project. To a C coder, invocation 



LISTING 3. How to invoke the JVM in an application using JDK 1.2. 



II JDK 1.2 Invocation example 
♦include <jni.h> 

// Setting some standard options. 

extern jint JNICALL PrintfC FILE *f, const char *fmt, »a_list args ); 

extern void JNICALL Exit( jint code ); 

extern void JNICALL Abort( void ); 

♦define NOPTIONS = 5; 

Jai/aVHDption DPTI0NS[5] = 

{ 

{ "classpath", ( void*) "C: \\Java\lib\classes.zip; D:\Game\classes" }, 
{ "verbose", (void*)"jni,gc" }, 
{ "vfprintf", (void*)Printf }, 
{ "exit", (void*)Exit }, 
{ "abort", (void*)Abort } 

}; 

// Create a JDK 1.2 JavaVH as desired. 

JNIEnv* SV_InitJavaVH( JavaVMOption* options, jint nOptions ) { 
JavaVHInitArgs vm.args; 
JavaVH* vm.handle; // not preserved 
JNIEnv* env; // return to caller 
jint ret; 
// Request version 1.2 
vm.args. version = JNI.VERSION.1.2; 
ret = JNI_GetDefaultJavaVHInitArgs(&vm_args); 
if ( ret==0 ) { 

vm_args. options = options; 

vm.args. nOptions = nDptions; 

vm.args. ignoreUnrecognized = JNI.TRUE; 

ret = JNI.CreateJavaVH( fevm.handle, Sienv, ftvm.args ); 

if ( ret==0 ) { 
return env; 

} 

} 

... // error handling 
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Java wraps native code in Prax WAR 



Written by Billy Zelsnacl< 
during his days at 
Rebel Boat Rocl<er, 
Prax War was destined 
to be the first major game to use Java for 
most of its code. With the exception of a C 
renderer, the Prax War engine was written 
entirely in )ava. Zelsnacl< explained the 
game this way: "We use JDK and JNI. The 
game itself starts from Java, f use Java as 
controller code for C. Java is very good at 
calling C code, but [it is] not necessarily as 
clean the other way around." The design 
l<ept raw data (such as textures and 
sounds) on the native side, but made 
them accessible to Java as needed. Billy 
Zelsnacl< found few problems with the 
core classes (at one point, UDP networl<- 
ing performance was an issue), and found 
no problems with the most feared Java 
component, the garbage collector. The 
engine used just two threads — one 
thread that listened for incoming pacl<ets, 
plus the main loop itself. 

Unfortunately, Rebel Boat Rocl<er's 
publisher, Electronic Arts, decided earlier 




this year to cancel the project, stating that 
the game had "missed its technology win- 
dow". If Electronic Arts had had the Idnd 
of faith in Rebel Boat Rocl<er that Sierra 
has shown in Valve Software, we might 
have found that their assessment was 
straight to the point — maybe Prax War 
missed its technology window by being 
too early. 

Zelsnacl< summed up his Java experi- 
ence up this way: "Java opened up possi- 
bilities for the product that could not have 
been realized without its power. It was 
one of the things I was most excited about 
and proud of." 



might seem a natural choice. This 
architecture Is known as embedded 
Java. Your application Is linked to a 
.DLL that provldesaJVM , which hap- 
pily lives and dies within your applica- 
tion, completely at your disposal. It 
looks like this: 

// Engine piggy-backed with Jl/M or 
// Engine retrofitted with JVM 
// Set options, possibly parsing commandline. 
nOpt = SI/_GetOptions(8ioptions,argc,arg»); 
// Invoke Jl/H, get script engine started. 
SV.StartVHC SV.InitJa«aVH(options, nOpt)); 
// Start the actual game, 
return gamel"lain( argc, argv ); 

On the Other hand, if you write a 
purejava game, or use the 
somegame.GameMain class shown In Listing 
2, then some other application loads 
theJVM and hands it thejava bytecode 
of your game. This scenario Is used 
when a web browser runs "gamelets", 
for example, or when J DK's ja»a loads 
an executable. JAR file. Whether your 
game uses native code or not, you do 
not have to concern yourself with 
invocation if themain loop Iswritten 
in Java. Native method code will be 
en capsu I ated i n J ava cl asses, as I on g as 
the .DLLs required are loaded in time. 
It does not look like much of a differ- 
ence, but choosing one or the other 
might have a huge impact on your 
project. 



Embedded Java: A Natural Choice? 

Let's look at an examplethati call the 
"Quake 3 scenario." Your team has 
nearly finished agameenginewritten in 
C orC-H-. The game has a large and sta- 
ble legacy code base that you don't want 
to tamper with, yet there is a clear-cut 
need that J ava might ad dress, such asa 
new server-side scripting language, or 
support for client-down loadable code. In 
short, you want to retrofit an existing 
application with a J ava component. 

The history of the Quake engine is a 
great example. Quake featured a custom 
scripting language (QuakeC), Quake 2 
introduced a server-side .DLL 
(GAME.DLL), and someQuAKE engine 
offspring now deploy client-side .DLLs. 

Embedding Is possibly the best 
answer In all cases where you have to 
deal with C legacy code that Is not 
implemented In an object-oriented fash- 
ion. TheJVM Isjust another device that 
is initialized, configured, started and 



shutdown again. There are some restric- 
tions (for Instance, you cannot restart 
theJVM once it has shut down), but In 
general, all you do is provide raw data 
(bytecode) to the embedded JVM much 
thesameway you'd feed .WAV files to a 
sound device. If you do not want to use 
.DLLs at all, embedding is your solution. 
You also get a lot more control over the 
JVM that Is used by your game. Ship- 
ping ajava Runtime Environment (| RE) 
with an embedded solution might save 
you support and maintenance 
headaches. It might also address some 
reverse engineering, tampering and 
cheating Issues. 

If embedded Java is used, either C 
control codeexecutesjava methods on 
theJVM which return thedata, or the 
Java code in turn calls native methods 
to writeback. You could havejava 
threads run in parallel to your applica- 
tion, but debugging an application that 
moves back and forth between native 
and Java execution stack frames can bea 
challenge to you and your tools — mul- 
tlplethreadswlll make it even tougher. 

What problems are specific to using 
an embedded JVM?Somehave already 
been mentioned, such as negligence or 
outright omission of the Invocation API 



from somejava implementations, and 
potential problems you might face 
when falling back on compiling J ava to 
native code. All of these problems can 
be overcome one way or the other, how- 
ever. The real danger might be much 
more subtle. 

Yourlegacy codehasacertain design 
— possibly not object oriented at all if 
you used C, or possibly an object-orient- 
ed design that maps badlytojavalf you 
used C-H- excessively (if you made use of 
templates and/or multiple inheritance). 
In thesecases, taking a single compo- 
nent of your game (for instance, the 
server-side game logic) and converting it 
to Java could introduce bugs and errors 
in formerly stable and tested code. 
Worse still, through JNI thedesign used 
in native code will proliferate in to J ava 
code, resulting in badly designed Java 
code. For example, if you never handle 
objects (see Listing 1 and itsuseof class 
methods), it Is unlikely that you are 
using an object-oriented design. Legacy 
code tends to share memory using 
pointers for speed and convenience, 
which Is not possible with JNI. You have 
to think hard and makejudlclouscuts 
to get a lean Interface between Java and 
native code. High levels of abstractions 
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implemented as abstract base classes 
and interfaces usually work best: the 
more details you hide, the better] N I 
will work for you. 

Consider handling structured data on 
both sidesof JNI, such as that used for 



LISTING 4. Java Proxy for a C++ object. 



collision handling. Collision responseis 
part of the game logic (does the player 
takedamage, bounce, or die?) and isthus 
handled in thejavacodein our example. 
Collision detection might be performed 
within thescenerepresentation that is 



also used by the rendering code- 
almost surely native code you want to 
keep. In thisscenario, yourjavagame 
I ogi c m i gh t cal I n ati ve code to trace an 
object's movement through thescene. 

Thisiswherethelevel of abstraction is 
rela/ant. TaketheQuAKE 2 representa- 
tion of a vector in 3D space: float[3] . I n 
J ava, th i s i s best represen ted as a cl ass 
with float x,y,z fields This avoids array 
bounds checking overhead, and frees us 
from worrying whetherJN I pinsor 
copies thearray. For objectsthat small 
and likely to have all their fields accessed, 
thesimplest way to pass them back and 
forth isto unfold them on the stack as 
primitive data types, much thesamethey 
would be flattened for serialization. 

This solution is more the exception 
than the rule, however. In general, it 
pays off to hide as much detail as possi- 
ble on both sides. The game logic does 
not need to know whether axis-aligned 
bounding boxes or spheres are used for 
collision detection, it only has to initi- 
ateupdatesto position and size. For the 
actual trace in native code, it is irrele- 
vant whether a given entity within the 
bounding volume is a player, a mon- 
ster, or a fireball. 

Using a high level of abstraction on 
the J ava side by sticking to abstract base 
cl asses an d i n terf aces m akes retri evi n g 
and caching method IDsin your native 
code much easier, si nee all objects with- 
in an inheritance tree will share the 
same signatures. You might find it safer 
to cachefield and method IDsin class 
descriptor structs or C-H-objects. Field 
access is more efficient, but exposes the 
internal implementation of your Java 
obj ecta JNI meth ods I i ke GetFieldiD an d 
GetFloatField can be used instead of, say, 
GetHethodID and CalLFloatHethod to access 
instance fields directly. 

You pass references as jobject handles 
instead of pointers to makejavadata 
accessibleto native code. The reverse is 
not possible: neither C structs nor C-H- 
objectsarevisibletojava. You can 
address C-H-objects or C structs with jint 
handles on thejavaside, using more 
(hash table look-up) or less (typecast) 
safe ways to retri eve the ef f ecti ve 
address. A proxy class would then wrap 
native methods with public accessors, 
I ike that sketched out in Listing 4. 

Of course, if you want to avoid 
switch statements in the native 
method, or you want to wrap C-H- 
accessor methods instead of exposing 



package jni; 
class Proxy { 

/** Handle to retriei/e C++ object, natii/e side. */ 

private final int handle; 

/*+ Native LUT/constructor. */ 

private static final native int neuNativeO; 

/*+ Constructor, gets/creates a handle to a native object. */ 

public Proxy { 
handle = neuNativeO; 

} 

/** Simplified fieldlD. */ 
private final int SOME. FIELD = 1; 
/** Accessor hiding the simplified retrieval. */ 
public final float getSomeFieldO { 
return getFloat( handle, SOME FIELD ); 

} 

/** Method that saves us many retrievalO functions. */ 
private final native float getFloat( int handle, int field ); 

} 

// Minimal C++ object, and JNI glue, 
class NativeObject { 
public: NativeDbject( jobject ouner ) { 
this. owner = owner; 

} 

public: jobject getOwnerO {return owner;} 
public: float someField; 
public: jobject owner; 

}; 

#include <jni.h> 

extern jclass InvalidProxyOwnerException; 
extern jclass InvalidFieldlndexException; 
// extern "C" implied 

JNIEXPDRT jint JNICALL Java_jni.Proxy.getFloat 
( JNIEnv* env, 
jobject owner) 
{ 

return (jint) (new NativeObject(owner)); 

} 

JNIEXPORT jfloat JNICALL Java.jni_Proxy_getFloat 
( JNIEnv* env, 

jobject owner, 

jint handle, 

jint field ) 

{ 

// Truly dirty. Trust on blank finals. 
NativeObject* obj = (NativeObject*)handle; 

if ( obj->getDwner() != owner ) 
(*env)->ThrowNew(env, InvalidProxyOwnerException, "access attempted by non-owner"); 

switch( field ) { 

// Enums to be kept in sync manually... 
case 1: return obj->someField; 
default: 

(*env)->ThrowNew (env, InvalidFieldlndexException, 
"access attempted by non-existing field"); 

} 

} 

} 
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fields, you could also implement the 
pu bl i c J ava accessor as a n ati ve meth od . 
Incidentally, maintaining the same set 
of enums i n J ava an d n ati ve code i s on e of 
the problems that does not yet seem to 
have an elegant solution. If you are 
using a look-up tableto retrieve point- 
ers for handles, thejobject argument 
might already be sufficient. 

A native proxy implemented asaC-H- 
objectorC struct could cachea global 
jobject reference along with method IDs 
and field IDs Caching actual game 
data inside native codemeansthat 
your proxies have to be kept synchro- 
nized with the master objects, or you 
will end up with consistency errors 
that are very difficult to track down. 

Alternatively, you could encapsulate 
the results or take a snapshot of a 
n ati ve obj ect's state in a n ew J ava 
object created in native code, using the 
JNI function NeuObjectO to call ajava 
constructor. This approach works even 
better if your native and Java modules 
communicate by passing event descrip- 
tor objects to a queue. 



Encapsulated Native: A Magic Bullet? 

If you go down the road of J ava con- 
trol code, be prepared to throw out 
legacy code whenever necessary. 
Encapsulation meansdividing and split- 
tingyour codebaseinto tiny pieces- 
heaven if you are at liberty to design 
from scratch, hell if you have to handle 
code that isjust sticking together. If you 
can't isolatenative code modules and 
wrap them with Java classes, then there 
won't be a secure migration path. 
Handling a native legacy code base 
might well take more time than gutting 
it and starting from scratch. If you con- 
sider abandoning C/C-H-asyour main 
language, an encapsulation architecture 
is definitely the way to go. 

JavaandJNI will always have some 
performance disadvantages that make 
them unsuitablefor time-critical inner 
loops. However, it can be very efficient 
write your control codein Java (which 
can account for up to 90 percent of a 
game'stotal code base) even though that 
often takes up less than 10 percent of 
the overall processing time— especially 
for games that make the CPU spend half 
ofthetimein an OpenGL driver. This 
was the reasoning behind thePRAX War 
architecture that Billy Zelsnack imple- 



mented at the now defunct game devel- 
opment studio. Rebel Boat Rocker (see 
sidebar, "Java wrapsnativecodein Prax 
War"). There are on ly a few areas i n 
which native code is really needed, such 
asman aging raw data (textures and 



sound resources, for example), and ren- 
dering. Collision detection might best be 
done in native code shared with theren- 
derer. Collision response, however, isa 
natural part of high-level game logic. In 
some cases, the lackluster performance 



id Abandons Java for QUAKE 3: ARENA 



John Carmack considered using 
Java in id's games for quite some 
time, ever since he announced that 
the company was leaning towards 
client-downloadable code for the 
Trinity project. "The QA game architecture 
so far has two separate binary .DLLs: one 
for the server-side game logic, and one for 
the client side presentation logic." Games 
that licensed the Quake 2 engine, notably 
Half-Life and Heretic 2, also wound up 
using client side .DLLs. However, with the 
hacking attacks on Quake 2 servers in 
mind, Carmack states that, "While it was 
easiest to begin development like that, 
there are two crucial problems with ship- 
ping the game that way: security and 
portability. If we were willing to wed our- 
selves completely to the Windows plat- 
form, we might have pushed ahead, but I 
want Quake 3: Arena running on every plat- 
form that has hardware-accelerated 
OpenGL and an Internet connection." 

His solution: "I had been working 
under the assumption that Java was the 
right way to go, but recently I reached a 
better conclusion. The programming lan- 
guage is interpreted ANSI C. The game 
will have an interpreter for a virtual RISC- 
like CPU." Unreal followed a similar 
approach: companies that license the 



engine can opt to use compiled C or C++ 
code, and interpreted UnrealScript is 
available for homebrew scripting. 

The advantages of using a C or C++ 
subset for your VM are obvious when it 
comes to handling legacy code. 
Ironically, it was Java portability prob- 
lems that led id to develop the Quake 3 
custom VM. Sun's promise of "write once, 
run anywhere" did not hold for the 
Invocation API on important server plat- 
forms, so Carmack decided to abandon 
the embedded JVM he had planned to 
use. "My ideal situation," he stated, 
"would be to include the interpreter in 
the QUAKE3.EXE, and just treat class files 
as data to be loaded and dealt with tike 
anything else." Unfortunately, while this 
solution works fine on Win32 platforms, 
this was not guaranteed for Linux, OS/2, 
or even Solaris. "Having made the deci- 
sion to do my own interpreter, I feel much 
more at ease not having to rely on anyone 
else's external code. When it comes 
around to the next development cycle, I 
will make the Java decision again." As for 
embedding: "We are still working with 
significant chunks of an existing code 
base. If I did want to go off and start 
fresh, I would likely try doing almost 
everything in Java." 
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of Java core cl asses mightforce you to 
replace them with your own custom Java 
code, or even use some native code 
instead. In theend, you will haveafew 
cleanly separated native code modules 
you can optimize to your heart's con- 
tent, controlled by robust Java code. The 
object-oriented design propagates top- 
down into your native code, which 
should be another benefit. 



The Holy Grail: 100 Percent Pure 
Java 

Supporting the multitude of 
Internet server platforms (Solaris, 
Linux and other UNIX flavors, OS/2, 
and Windows NT) has become 
increasingly important for multiplayer 
games. Presuming the existence of 
decent Java networking core classes 
and acceptable performance of Java- 
based scene lookup and collision 
detection code, a dedicated server 
implemented entirely in Java is an 
attractive possibility — portable by 
default and, in theabsenceof JNI, eas- 
ily compiled to native code. 

Portability issues are not as pressing 
for clients, the majority of which are 
W i n 32-based . At th e same ti me, a real 
need for native code might only be 
found fortheclient, which ispushing a 
lot of raw data (textures and sounds) 
from local disk to local memory to 
nativedriver code. Unfortunately, only 
a few games, such as id's experimental 
QuakeWorld release, have separated 
the client and server completely. 
Consequently, a dedicated Java server 
means a separate code base that partly 
duplicates the shared client/ server 
sources. Dedicated servers have become 
quite common recently, but in the 
long run, thecodeduplication isnot 
acceptable. Automated Java-to-C or C- 
to-Java conversion might offer a tem- 
porary workaround only. 

Ultimately, shipping a client written 
in Java will require decent Java bind- 
ings around reliable, cross-platform 
APIs, and these are nowhere to be 
found. Java does not have official 
OpenGL bindings, there is not even a 
portable native API for 3D sound, and 
the politics surrounding Sun's propri- 
etary J avaBD scene graph API doesn't 
help matters. For the foreseeable 
future, commercial games will not be 
feasiblewithoutJNI and native code. 



QZJava Today: What's Next? 

Few games (Red Storm Entertain- 
ment's Politika is among them) 
have shipped with Java built into 
them, but if you want to see a full -si zed 
example of an embedded Java VM run- 
ning Quake 2 death match right now, 
you should visit theQ2jAVA web site at 
http://www.planetquake.com/q2j ava. 
Q2JAVA, orginally by Barry Pederson, is 
a cooperative open source implementa- 
tion of the Quake 2 multiplayer game 
logic and works with the native Quake 
2 executableson Windows 95/98/NT, 
Linux and (asa dedicated server) 
Solaris. 

Thisarticlehasintroduced thetwo 
maj or roads to usi n g J ava for your game, 
though admittedly, a lot of details have 
been omitted and major issues (like 
security) were not touched upon. 



However, Gamasutra.com is hosting 
more of my J ava game development 
articles, including a web version of this 
articlethat includes an annotated list of 
references. ■ 



Embedded Java In Vahpire: The Masquerade 



In a recent developer update, 
NIhlllstlc's Director of Tech- 
nology, Robert Huebner, stated: 
"I've always been rather antl- 
java; all the Internet hype surrounding 
the language was overwhelming. But 
after examining the language further, it 
was clear that it makes an ideal scripting 
language for games. The embedded Java 
API allows us to provide our designers 
with a subset of the Java environment, 
and the JNI interface allows us to pro- 
vide hooks from the Java Virtual Machine 
(JVM) directly into the game engine." 
The new java-based system will replace 
a custom compiled language, COG, that 
the team used in its previous title, Jedi 



Knight: Dark Forces 2. According to 
Huebner, "The JVM is a lot faster than 
the systems we wrote ourselves; their 
kernel is more heavily optimized, the 
available Java compilers produce much 
more optimized object code, and the 
newest JVM systems include just-in-time 
QIT) compilation to native instructions 
as a standard feature. And since the lan- 
guage is so much richer than our previ- 
ous C-subset, it gives the designers a 
much wider range of possibilities." 
Because Nihilistic is developing primari- 
ly for Windows, it is able to apply solu- 
tions that were not feasible for id 
Software's multi-platform Quake 3: 
Arena strategy. 




• See the JNI specification at: 
http://java.sun.eom/products/jdk/1.2/ 
docs/guide/jni/index.html 
http://java.sun.eom/products/jdk/i.i/ 
download-pdf-ps.html 

• See Rob Huebner's 1999 GDC slides at: 
http://www.nihilistic.com/GDC99/Java. 

• Gordon, Rob, et al. Essential JNI: Java 
Native Interface. Prentice Hall Computer 
Books, 1998. 

• See the extended version of this arti- 
cle on Gamasutra.com, which includes 
further links and resources. 
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n my article last month, I covered the 



basics of curved surfaces, including 



Her mite curves, Bezier curves, and Bezier 



surfaces. I warned that the implementa- 



tions would be straightforward and naive. 



and therefore slow. 

That waslast month. Thepurpose of thisarticleisto take 
what we learned last month and delve into optimization 
strategies and algorithms so that our slow, straightforward 
codecan become fast (but still straightforward) code. 

This article ends with a more interesting demo than last 
month's — a terrain system of Bezier patches. Therefore, this 
article presents information that bridgeslast month'sinfor- 
mation and this month's terrain system. We'll find a better 
patch tessellator, quickly talk about terrain light map genera- 
tion and why it'sa good idea, and finally talk about putting 
patch es togeth er to form th e terrai n an d th e bevy of com pi i - 
cationsthat accompany it. 

Central Difference Tessellation 

At the end of the last article, I created a demo of a single 
Bezier patch, tessellated uniformly, bright red, that 
spun around at a woeful frame rate. Certainly, this was 
nothing you could use to build a robust application. The 
computations involved for just that one patch included 32 
cubic function evaluations, 32 quadratic function evalua- 
tions, two vector normalizations, a vector cross product, and 
OpenGL lighting for each point. Wecan do two things to 
speed that up: do less work per point, and, even more 
importantly, calculate fewer points. 

Perhapsthe worst sin of my naive implementations was 
thepatch tessellator, UniformPatchTessellator. It just calculated 
a ten-by-ten grid of points on thepatch. It calculated each 
point explicitly and just about as slowly as it could. If we can 
devise a better patch tessellator, we're a long way towards a 
truly useful implementation. 

There are better tessellation algorithms out there, and I'll 
discuss one particular algorithm that I've used to make these 
Bezier patches really crank: central differencing. Oneform of 
thecentral differencing algorithm ismentioned in Watt and 
Watt's Advanced Animation and RenderingTechniques 



W hen Brian's not coding up a storm or sleeping through meet- 
ings at CogniToy, he's filling up otherwise useful pages of the 
magazinewith his nonsensical ramblings. Keep him busy by 
writing to him atbrian_sharp@cognitoy.com with questions or 
comments or else he might find the time to write again. 



Rounded landscapes like this can be rendered quicldy using 
Bezier patches. 
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(Addison -Wesley, 1992), but this version is modified signifi- 
cantly from theonethey present. 

The fundamental mathematical construct used in central 
differencing istheTaylor polynomial. ATaylor polynomial 
of a function is a polynomial that approximates that func- 
tion near a certain point. So, let'ssay you've got a sine curve, 
and you start atop a peak of the curve. Then, you can form 
polynomialsthat look like a sine curve near that peak. As 
the polynomial becomes more and more complex, it looks 
more and more I ike that sine curve. The formula for the 
Taylor polynomial of a function h(x) near a parameter u is: 

h(u + du) = ^^^^h'(u) where h' is the i* derivative of h. 

Then, in practice, you pick some limit other than infinity 
for the summation, and you get an approximation of the 
function — thehigherthelimit, the better the approxima- 
tion. However, in our case, we're dealing with cubic func- 
tions Therefore, any derivative after the third will be zero, 
so it's pointless to maketheupper bound anything higher 
than three. So, if our cubic function ish(x), its Taylor poly- 
nomial will look likethis: 



h(u + du) = h(u) + du(h'(u))- 



du^ , du^,, 

+ h (u) + h 

2 4 



'(u) Eq. 1 



You've probably noticed by now that all thisdoes istake 
one cubic polynomial and give us another cubic polynomial. 
You're probably wondering what that does for us. This is 
where central differencing takes over. The idea behind cen- 
tral differencing isthat, given information about theend- 
pointsof a curve segment, we'd I ike to be able to find the 



FIGURE 1 . The parameters used in the Taylor polynomials 
for central differencing. 




midpoint. That way, we can then recurseon both sides of 
the midpoint and find their midpoints, and so on, until we 
have our curve. 

Rather than keep you hanging, I'll mention theprocess 
right now and go through the derivation of it afterwards. If 
wehavetwo points alongthefunction h(x), say h(a) and 
h(b), and we want to find the point halfway between h(a) 
and h(b), or h((a-i-b)/2), we do thefollowing. First, weaver- 
age h(a) and h(b), which gives us just a point on the line 
between them. 

Next, wetakeh"(a) and h"(b), thesecond derivative of h 
ata and b. Weaveragethem to find thesecond derivative 
at the midpoint, h"((a-i-b)/2). We also find du, which is half 
the parameter distance between h(a) and h(b), or (b-a)/2. 
Then, we compute -du^ *h"((a-i-b)/2) / 4. We add that to the 
averaged point we found above, and that's our point. The 
pseudo-codeforthisprocessisshown in Listing 1. 



LISTING 1 . This code tal<es two points on a cubic curve and 
the second derivatives of the curve at those points and finds 
the midpoint and its second derivative. 



CentralDifference(h(a), h"(a), h(b), h"(b)) 
{ 

// Get the midpoint parameter value, 
mid = (a+b)/2; 

// Get the midpoint of the line betueen 

// h(a) and h(b) 

avgPoint = (h(a) + h(b)) / 2; 

// Find the second derivative of 
// h((a+b)/2). Since the second 
// derivative is linear, ue can 
// just average them. 
h"(mid) = (h"(a) + h"(b)) / 2; 



Central Differencing: The "Why" 



So we've established that central differencing is a better 
way to tessel late a patch. Now we need to choose some 
valuesso that theTaylor polynomial makes this work. We'll 
call themidpoint h(u), and du is the distance from themid- 
point to an endpoint. That makes thefirst endpoint h(u-du), 
and the other endpoint h(u-Kiu); Figure 1 shows an example 
curve. From that, we can evaluate theTaylor polynomial as 
though we were solvingfortheendpoints, given themid- 
point. That gives us the following two equations: 



h(u + du) = h(u) + du(h'(u))- 



h(u-du) = h(u) + -du(h'(u)) 



du' 



h"(u)- 



du" 



(-du)' 



h"(u) + 



h'"(u) 



(-du) 



Eq.2 



-h"'(u) 



// Find du, the distance from h(a) 
// to h((a+b)/2). 
du = (b-a) / 2; 

// Now h(mid) is this next term 
// plus avgPoint. 
secondTerm = -(du*du)*h"(mid)/4; 
h(mid) = avgPoint + secondTerm; 

return h(mid) and h"(mid) 



Eq. 3 



In Equation 3 we can pull the negative sign outof the-du 
terms on the right side, giving use these two equations: 



h(u + du) = h(u) + du(h'(u)) + ^h"(u) 



+ h 

4 



•(u) 



h(u-du) = h(u) + -du(h'(u))- 



du' 



h"(u)- 



-(du' 



-h'"(u) 



Eq. 4 



Eq. 5 



http://www.gdmag.com 



JULY 1999 GAME DEVELOPER 



CURVED SURFACES 




Equations 4 and 5 by themselves do not give us enough 
information to soivefor the midpoint ti(u), but we can reduce 
them to a single nice equation just by adding them together. 

h(u + du) + h(u-du) = 2h(u) + du'h"(u) ^ ^ 

Eq. 6 

Thepositivedu and negative du terms cancel out, which is 
why wehave such a nice equation. However, we don't want 
to soivefor theendpoints— weneed themidpoint. 
Rearranging the equation we get: 

2h(u) = h(u + du) + h(u-du)-du'h"(u) ^ ^ 

Eq. 7 

Dividing through by 2 and breaking things up, it becomes: 
h(u + du) + h(u-du) du'h"(u) 



h(u 



Eq. 8 



That first big fraction on theright sideof theequation is 
just the average of the two endpoints That'seasy enough to 
find because we're assuming wehave whatever information 
weneed about theendpoints Thesecond fraction on the 
right side is a little harder, though. It depends on thesecond 
derivativeat the midpoint. Can we get that from theend- 
point? Well, note that thesecond derivative isjust a linear 
equation. That'seasy enough to find from theendpoints; we 
can just average the second derivatives of h at the endpoints, 
and we've got it: 

,„, . h"(u + du) + h"(u-du) 

h (u) = — ^^r— Eq. 9 



Now, throwing that back into equation 8, we've got: 
h(u + du) + h(u - du) du^(h"(u + du) + h"(u-du)) 



h(u 



Eq. 10 

That's a mouthful, butthat'sit. What that says isthat we 
can find the midpoint by averaging the endpoints and 
adding on a weighted average of theendpoints' second 
derivatives. Since we know we can find the midpoint's sec- 



ond derivative (we just did it in Equation 9) we'll have 
enough information when we're done to recurseand find 
the rest of th e cu rve. 

That'sjust for a curve, though, ash isonly a function of 
onevariable. We want do thisfor patches, which are func- 
tions of two variables. But before we move on, we should 
take note of an interesting property of the central difference. 



Nonlinearity 



In Equation 10, we find the midpoint by averaging the 
endpoints and adding on the other term, the weighted 
second-derivative term. What's interesting isthat if that 
second term isO, then it means that the midpoint lies on 
the line between theendpoints. Therefore, we'll refer to 
that second term as the nonlinear term of the central differ- 
ence — it determines how far the midpoint isfrom the 
line between theendpoints. 

As you might suspect, this is a great heuristic that we 
can use to decide how far to tessellatea curve. Wejust set 
a certain threshold, and when the magnitude of the non- 
linear term is below that threshold, we stop recursing and 
return. Therefore, we'll have much more detail in very 
curved areas, and not nearly as much detail in the less 
curved areas. This is a perfect way to save detail for the 
parts of the curve that need it. We'll usethenonlinear 
term later on when we put together heuristics for our 
patch tessellation. 



Central Differencing Revisited 



We've derived enough to tessellatea curve with cen- 
tral differences. Now weneed to figure out how to 
tessellatea patch with central differences. As it turns out, 
central differences in two dimensionsare slightly harder 
than the one-dimensional variety. So, we describe our 
problem again. This time, we want to be able to take infor- 
mation about four corner points on a patch and find the 
same information about the midpoints between the cor- 
ners and also the center point. An illustration of this is 
Figure 2. 

Looking at the illustration, it's clear that if we can find 
the midpoints of this one level given the corners, we can 
recurse within each of thefour smaller squares using the 
same method, and so on, until we have our patch. Now, we 
aren't sure exactly what information we need at the corner 
points, so let's see what it would take to find the other five 
points. 

Theu-midpointscan be generated using the one-dimen- 
sional central differencing. We can ignore v, since it stays 
constant, and use theendpoints and theirsecond deriva- 
tives with respect to u to find the u-midpointsand their 
second derivatives with respect to u. 

The v-midpoints can be generated thesameway. Wecan 
ignore u and use the endpoints and theirsecond deriva- 
tives with respect to v to find the v-midpoints and their 
second derivatives with respect to v. 

To keep this all straight, it's useful to keep track of what 
information we have for which points: 
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corners^'f (u,v! 



u-mids^f (u,v), 



v-nnids-> f (u,v), 



a'f (u,v) 9'f (u,v) 

a'f(u,v) 



a^f(u,v) 
av^ 



center nothing 

Before we worry about finding the center point, we should 
mal<e sure the u-midpoints and thev-midpoints are complete. 
That is, if we want to recurse, we need to make sure that we 
end up with thesame exact information atthenew points as 
we got from the corner points. Otherwise, that information 
won 't be aval I abl e to th e four smal I er squares wh en we try to 
recurse. So, the objective is to find the second partial deriva- 
tive of f with respect to v at the u-midpoints, and the second 
partial derivativesin u at thev-midpoints. However, it'snot 
clear quite how we do this, since the one-dimensional central 
differencing doesn't say anything about how to interpolatea 
variableofv along u or vice-versa. A clue comes from the 
equation of the Bezier patch: 

3 3 



PijBi^(u)Bf(v) 



EL 

i=0 j=0 

Let'sfind thesecond partial in v along u first. We must find 
the second derivative of the patch with respect to v. From the 
above equation, we'd have: 

3 3 d^Bf(v) 

i=0 j=0 



dv' 



Now, thebaslsfunction with respect to u, B(u), isacubic. 
Since it's untouched, we can seethat the second derivative of 
thepatch with respect to visa cubic function in u. If wehave 
any cubic in u, we can interpolateit in theu direction using 
the one-dimensional central differencing. 

In thiscase, then, thefunction we're interpolating isthe 
second derivative with respect to v: 

<3'f(u,v) 



g(u) = 



So, we can interpolate g(u) if wehave g(u) attheendpoints 
and g'ssecond derivative with respect to u attheendpoints. 
That means that to get thesecond partial derivativein vat the 
u-midpoints, thecorner pointsneed thisdata: 

<?'f(u,v) d^'Hu.y) 



The corners already have the first value, thesecond deriva- 
tive with respect to v. We have to add thesecond one, thesec- 
ond partial derivative with respect to u of thesecond partial 
with respect to v. 

Now we've got thesecond partialsin vat the u-midpoints. 
Wecan easily find thesecond partialsin u at the v midpoints 
in thesame way if wejust swap the variables. So, thesecond 
partial in u isacubic function in v, so wecan find it by adding 
th e fol lowi n g to th e corn er poi n ts: 

(9'f(u,v) d''f(u,v} 



Again, the corners already havethefirst item. Also, we can 
switch the partial s around, so: 



Eq. 11 



Finally, we note that this term isa linear function of u and 
v, so wecan get it at the u-midpoints and thev-midpoints just 
by averagi ng the val ues from th e corner poi nts. 

Therefore, wehave all the information weneed in thecor- 
ners, and wecan get that information back at the u-midpoints 
and v-midpoints. Just to keep track, here's what we're up to: 

<9'f(u,v) ^i'f(u,v) d^Hu.v) 



corners -> f(u,v) 

u-mids^'f (u,v), 

v-mids^f (u,v), 
center -> nothing 



dHju.v) dHju.v) ^"f (u,v) 

ai' ' a/' ' ai'a/' 

<?'f(u,v) d^f(u,y) <?f (u,v) 



Now, wejust have to fill in thecenter point, and we're 
done. (Don't worry, we're almost there.) 

Referring back to Figure 2, the center point can bederived 
from theu-midpointsjust aseach of thev-midpoints was 
derived from its two corner points. So we use the u-midpoints 
and their second partialsin v to find thecenter point and its 
second partial in v. Then, wecan get the center's second par- 
tial in u by averagingthev-midpoints' second partialsin u. 
Finally, wecan get thecenter point's final value, thesecond 
partial in v of thesecond partial in u (thevaluefrom Equation 
11) by averaging the values from thev-midpoints. 

We're done! Ittookawhile, but we finally have all the 
information for all the points: 

<9'f(u,v) <9'f(u,v) d^fiu.y) 



corners -> f(u,v) 
u-mids^'f (u,v) 



aj' ' a/' ' a'a/' 

d^Hu.y) <?'f(u,v) d'i(u,v) 



v-mids^f (u,v), 
center f (u,v 



a' ' a/' ' a'^' 

<?'f(u,v) d^Hu.y) <?^(u,v) 



a' ' a' ' a'a' 

d^Hu.v) d^Hu.v) d^fiu.y) 



a^ 



a^ 



aw 



Therefore, given four corner points with those five values, 
wecan create theu-midpoints, v-midpoints, and center point 
with thecentral differencing. From there, wecan recurseinto 
thefoursmaller squares formed by thenew points to contin- 
ue tessellation. Pseudo-code for doingthisisshown in Listing 
2. We start by explicitly computing this information for the 
four patch corner points, and then recurse. That'sgreat, but 
there'sstill something missing. We've proved the recursive 
step, but we don't have a base case. When do we know we're 
done? How far do we recurse? 



Recursion Heuristics 

Weal ready mentioned a very powerful tool for deciding 
when to stop recursing. Thenonlinearity term from 
thecentral differences is an estimation of how curved thesur- 
face is at the spot we're tessel latin g. Wecan usethenonlinear- 
ity terms from thegeneration of the midpoints to determine, 
then, th e curvature of the subpatch we're working with. Then, 
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FIGURE 3. The nonlinearity vector 
lies roughly in a slice of the frustum. 



[ 



nonlinearity 
vector 




given some threshold, if the magnitude 
of the nonlinearity termsforall the 
midpointswaslessthan thethreshold, 
we'd just leave the subpatch asthetwo 
triangles formed by the corners and 
return. So if thethreshold isvery high, 
we end up with a very low-detail patch 
of just two triangles. If thethreshold is 
too low, the patch is extremely detailed, 
which will look good but run slowly. 
Hence, picking a good threshold value is 
just a matter of testing, and would be a 
good candidate for an options screen for 
the game player. 

However, just because a surface is 
very curvy doesn't mean we want it to 
continue tessellating. If the surface is 
50 miles away from the camera and 
occupiesathree-by-three block of pix- 
dson thescreen, we'd rather it be very 
low detail. However, if we zoomed in 
with a sniper rifle (think Golden eye) to 
look at that terrain, it might be 50 
miles away, but might now occupy a 
200-by-200 block of pixels, in which 
case we'd I i ke i t to tessel I ate q ui te a bi t. 
So, we need a way to add distance and 
field-of-view angle into our heuristic, 
and we'd like it to be fast. Ideally, we'd 
like to know how big the nonlinearity 
vector is in pixels. 

Here'sa good, albeit rough, heuristic. 
Consider the camera frustum. The non- 
linearity vector lies roughly in some 
slice of the frustum perpendicular to 
thescreen (see Figure 3). Let's say we 
can figure out how wide, in pixels, that 
slice of frustum is. Then the ratio of the 
magnitude of the nonlinearity vector 
to the width of thefrustum slice isthe 
(very rough) sizeof the nonlinearity in 
screen space, expressed as a percentage 
of thescreen width. For instance, if the 
patch was so curvy that thenonlineari- 



ty vector stretched all the way across 
that slice of frustum, the ratio would 
be 1. More likely, the nonlinearity vec- 
tor will be something relatively small, 
and the ratio will be something like 
0.002, which is 0.2 percent of the 
screen, equal to about five pixels in a 
640x480 screen display. 
However, to do this, we still need to 



find the width of that frustum slice. 
So first we take the vector from the 
eye to a point near the nonlinearity 
vecto r, like one of thecorner points. 
Call that vector d, for distance. Next 
we need the field-of-view angle, and 
then we're all set to do a little 
trigonometry. Referring to Figure 4, 
we know oneangleof thetri angle, 
and d is one of the legs, so the other 
leg ishalf thewidth of thefrustum 
slice. So that leg's length is d * 
tan(fov/2). Doublethat, and we have 
thewidth of thefrustum slice. Then 
take the ratio of the nonlinearity vec- 
tor's magnitude to thefrustum slice 
width, and we have our heuristic. 

Keep in mind that all of this isvery 
rough. Since it'sall about the visual 
quality, all of these heuristics are very 
open to tweaking. For instance, in my 
terrain demo, I decided that I cared 
more about detail of terrain close to 
the camera than far away, so I squared 
the distance attenuation partsoitfdl 
off faster and I could devote more tri- 



LISTING 2 . This code takes four patch corners and a number of derivatives at those 
corners and finds the midpoints and center points. 



II Find the midpoints and center point gii/en these four corners. Any point has four 
// values, uOi/0 is the point (the zeroth derivatives u/ respect to u and v, u2v0 
// is the second partial in u, u0v2 is the second partial in v, and u2v2 is the 
// second partial in u of the second partial in v. 
DifferencePatch(cO, cl, c2, c3) 
{ 

// Here are the points we're trying to find, 
points umidO, umidl, vmidO, vmidl, cen; 

// Find the u-midpoints' uOvO and u2v0 through Central Differencing. 

umidO.uOvO and uinid0.u2v0 = CentralDifference(c0.uOvO, c0.u2v0, cl.uOvO, cl.u2v0); 

umidl.uOvO and umidl.u2v0 = CentralDifference(c2.u0v0, c2.u2v0, c3.u0v0, c3.u2v0); 

// Find the v-midpoints' uOvO and u0v2 through Central Differencing. 

vmidO.uOvO and vmid0.uOv2 = CentralDifference(c0.uOvO, c0.uOv2, cl.uOvO, cl.uOv2); 

vmidl. uOvO and vmidl. uOv2 = CentralDifference(c2.u0v0, c2.u0v2, cB.uOvO, c3.u0v2); 

// Find the u-midpoints' u0v2 and u2v2 through Central Differencing. 

umid0.uOv2 and umid0.u2v2 = CentralDifference(c0.u0v2, c0.u2v2, cl.u0v2, cl.u2v2); 

umidl.uOv2 and umidl.u2v2 = CentralDifference(c2.u0v2, c2.u2v2, c3.u0v2, c3.u2v2); 

// Find the v-midpoints' u2v0 and u2v2 through Central Differencing. 

vmid0.u2v0 and vmid0.u2v2 = CentralDifference(c0.u2vO, c0.u2v2, cl.u2v0, cl.u2v2); 

vmidl. u2v0 and vmidl. u2v2 = CentralDifference(c2.u2vO, c2.u2v2, c3.u2v0, c3.u2v2); 

// Now we just have to find the center point. Find it from the midpoints. 
cen.uOvO and cen.u2v0 = CentralDifference(vmid0.uOvO, viiiid0.u2v0, vmidl. uOvO, 
vmidl. u2v0); 

cen.u0v2 and cen.u2v2 = CentralDifference(umid0.uOv2, umid0.u2v2, umidl.uOv2, 
umidl.u2v2); 



// We're done! 



}x 
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angles to thepatchesup front. In addi- 
tion, while it's possible to sit down and 
hash outtheideal threshold value for 
different performance levels, you'll 
probably have more luck (and spend 
less time) just making thethreshold 
modifiableat run time and tweaking it 
to suit your needs. 



Filling in the Cracks 

On e of th e dark secrets of th i s 
dynamic tessellation that we 
haven't mentioned isthat it causes 
cracking in the patch. When two sub- 
patch es th at sh are an edge are tessel I at- 
ed differently, there will be a hole 
between them, as shown in Figures. In 
practice, thecracking isvery evident 
and distracting, so it can't just be 
passed off as an acceptable visual glitch. 

At first glance, one might be tempted 
to go with a naive fix, likejust filling the 
cracks with triangles. The problem is, if 
the higher-detail surface is, say, two lev- 
elsof detail higher, theholewon't be 
triangular, but pentagonal. It turns out 
that it's easier and faster to fix the cracks 
in a better way than that. 

Cracking is caused when subpatches 
that share an edge are of different lev- 
els of detail. Therefore, the fix for the 
problem isto make it so that shared 
edges are always at the same level of 
detail. In my terrain demo, I use a post- 
pass over th e verti ces to accom pi i sh 
this For each patch, I check to make 
surethat the edges of its four subpatch- 
es are of the same level of detail. If one 



FIGURE 5. An edge shared by sub- 
patches at different detail levels 
exhibits an open triangular hole. The 
z value of the extra point in the red 
patch doesn't line up with the blue 
patch's edge. 




of them isn't at the same level of detail, 
I fix it by collapsing the middle vertex 
of the higher-detailed edge into the 
corner, as in Figure 6. Then I recurseon 
the four subpatches. Whilethisdoes 
require another pass over the data, it 
works surprisingly well. 

N o w th at we are tessel I ati n g patch es 
dynamically with no cracking, we're 
getting closer. But we're not there yet. 
Asfarasweknow, thepatch isstill a 
flat, unlit color. Next, we'd like to make 
it look a bit better. 



Textures and Light Maps 

On e of th e best ways to i m prove 
the visual quality of a patch is 
through texturing and lighting. The 
first thing we need are texture coordi- 
nates so we know how to place thetex- 
ture onto the patch . The simplest 
approach, sufficient for many purposes, 
isjust to use the patch (u,v) as the tex- 
ture coordinates. We can get these easi- 
ly from thecentral difference tessella- 
tion by averaging the (u.v) of the 
corners to get those of the midpoints 
and center point. 

Given those, it'seasy to drop atex- 
tureonto thepatch. We just toss a ter- 
rain texture of choice at OpenGL, and 
then use thepatch {u,v)ateach point as 
texture coordinates. This maps the tex- 
ture directly across the surface. But a 



texture with no lighting still doesn't 
look very good. 

It might be tempting to useOpenGL's 
lighting as wedid with the 
UniformPatchTesselLator. However, there are 
a number of reasons not to do this For 
one, finding the normal to the surface 
at each point requiresafairamount of 
work, including thetwo vector normal- 
izations and cross product. Furthermore, 
thecentral differencing is already com- 
plicated enough without having to 
interpolate thenormals. And there's 
one more reason not to use OpenGL 
lighting: OpenGL's per-vertex lighting 
makes the dynamic tessel I ati on of our 
terrain very obvious When the detail 
level oftheterrain changes, the lighting 
shifts disturbingly, and when the poly- 
gon count in theterrain is low, the 
lighting looks stretched and linear from 
theGouraud shading. 

Thesolution isto uselight maps (For 
a more thorough discussion of light 
maps, see"Multitexturing in DirectX 6 
byjason Mitchell etal. [September 
1998]). Light maps are handy not only 
because they not change with the tes- 
sellation, but they also actually give the 
illusion of more detail. Even ifthesur- 
faceismadeof two triangles, the light 
map still depicts smooth, curved light- 
ing playing across the curves that aren't 
really there. 

In my terrain demo, I didn't particu- 
larly want to pre-calculatemy light 
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LISTING 3 . This code uses Unif ormPatchTessellator and OpenGL feedback mode to 
make a light map for the terrain. 



II We want a huge projection so that it uon't clip anything, and ue uant 
// it orthographic to save on transformation cost. 

glMatrixMode( GL.PROJECTIDN ); 

glLoadldentityO; 

glOrtho( -10000, 10000, -10000, 10000, 1, 20000 ); 

glMatrixMode( GL.MQDELVIEW ); 
glLoadldentityO; 

gluLookAt( 0,0,10000, 0,0,0, 1,0,0 ); 



::giaear( GL.COLOR.BUFFER.Bn I GL.DEPTH.BUFFER.BH ); 

// Position the light. 

::glEnable(GL.LIGHTING); 

float position [] = {20. Of, 0. Of, 5. Of , O.Of}; 

::glLightf»(GL.LIGHTO, GL.PDSniON, position); 

// Tessellate our patch. 

generatePoints( size, controls, basesU, basesV ); 

// Kick OpenGL into feedback mode and render the patch, 
float* feedBuffer = new float[ size * size * 8 ]; 

: rglFeedbackBuff er( size * size * 8, GL.3D.C0L0R, feedBuffer ); 
: :glRenderMode( GL.FEEDBACK ); 

::glColor3f( 1, 1, 1 ); 

::glBegin( GL.POINTS ); 

for ( int V = 0; V < size; v++ ) 

{ 

for ( int u = 0; u < size; u++ ) 
{ 

: :glNormal3fv( norms + (u + (v*size))*3 ); 
::glVertex3fv( verts + (u + (v*size))*3 ); 

} 

} 

::glEnd(); 

// Read the data out of the buffer, 
int count = : :glRenderHode( GL.RENDER ); 

int texPos = 0; 

float* texData = new float [ 3 * size * size ]; 



for( int X = 0; X < count; x++ ) 
{ 

if ( feedBuffer [ x ] == GL_POINT_TOKEN ) 
{ 

texData[ texPos + ] = feedBuffer[ x + 4 ] 
texData[ texPos + 1 ] = feedBuffer[ x + 5 ] 
texData[ texPos + 2 ] = feedBuffer[ x + 6 ] 



I 




maps, and I didn't want to implement 
my own ligliting model, ather. 
Tlierefore, I found a liandy solution that 
generates fairly good-looking results: 
OpenGL feedback mode. 

The idea here is that at patch cre- 
ation time, we want to samplethe 
lighting at a number of points evenly 
spaced across the surface, and use their 
lit colors as tex els in a light map. It 
turns out that even afairly small light 
map will look just fine. Since we want 
the points evenly spaced, we'll use 
Unif ormPatchTessellator again to generate 
an 8x8 grid of points theslow way, 
light them, and build a texture out of 
it. Thecodeforthisisshown in Listing 
3. The process is surprisingly fast — a 
single patch's light map takes around 2 
or 3 milliseconds to generate and 
upload to OpenGL. 

Figure? shows theend product. We 
have the textured pass, the lit pass, and 
when combined, thetextured and lit 
patch. Ouitean improvement, indeed. 
If I'd promised a terrain system, though, 
a single patch ishardly sufficient. We 
n eed to take th i s good-l ooki n g patch 
and somehow come up with a way to 
define an entire landscape. 



Multiple Patches 



aking terrain out of multiple 
patches doesn't sound likeany- 
thing monumental. After all, we could 
just create a five-by-five grid of patches 
and draw them. Unfortunately, albeit 
not surprisingly, it's not that easy. 

The first and most obvious problem is 
that if wejust generate a 4x4 grid of ran- 
dom control points for each patch, the 



texPos += 3; 
X += 7; 

} 

else 
{ 

std::cout « "ERROR parsing feedback buffer array." « std::endl; 
deleteD texData; 
return 0; 

} 



Continued on p. 47. 



FIGURE i. For the curves to connect 
smoothly, their endpoint tangents 
must both point in the same direction. 
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LISTING 3. (continued from page 46.) 



II Hake it into a texture, 
unsigned int texNum; 
glGenTextures( 1, fetexNum ); 
glBindTexture( GL.TEXTURE.2D, texNum ); 

// Set the tiling mode. We don't uant lightmaps to repeat, or ue get odd borders. 
glTexParameteri(GL.TEXTURE.2D, GLJEXTURE.WRAP.S, GL.CLAHP); 
glTexParaiiieteri(GL.TEXTURE_2D, GL_TEXTURE_WRJP.T, GL.CLAHP); 

// Set the filtering. 

glTexParaiiieteri(GL_TEXTURE.2D, GL.TEXTURE.HAG.FILTER, GL.LINEAR) ; 
glTexParaiiieteri(GL_TEXTURE.2D, GL.TEXTURE.HIN.FILTER, GL.LINEAR.MIPNAP.LINEAR); 

gluBuild2DNipmaps( GL_TEXTURE_2D, 3, size, size, GL.RGB, GL.FLOAT, texData ); 

delete □ texData; 

return texNum; 



patches will lineup in x and y, butthez 
values of patches along shared edges 
will have nothing to do with each 
other. The terrain will have gaping 
h ol es wh ere patch es touch . 

Wecould make sure that the patches 
sh are th e same ed ge poi n ts so th at th ere 
wouldn't beany surface breaks between 
them. However, that's still insufficient — 
the Bezier patch representation doesn't 
guarantee anything about continuity 
between patches. We have to manually 
make sure that we preserve a number of 
conditions or else the seams between 
patches will likely be sharply discontin- 
uous, not at all like real terrain. 



Continuity Conditions 

eneed to en sure that the tan- 
gent vectors of patches along 
shared edges are the same, so that the 
terrain will look smooth, even between 
patches. A property of Bezier patches is 
that the tangent vectors at the edge 
points are defined by the edge control 
points and the control points one level 
in. So the tangent vector in theu direc- 
tion at the upper-right corner is the vec- 
tor through thecorner point from the 
point one to its left. 

To preserve continuity between 
patch es, we n eed to en su re th at th e for 
every control point along a shared edge, 
thetangent vectorsarethesame. The 
three points involved in those two tan- 
gents consist of the control point itself, 
plus the control points to either side of 
it. If we makesure that those three 



pointslieon aline, we know that our 
terrain will be smooth. 

Figures shows an exampleof this 
concept. Thetangent vectors of the 
joining curves pass through thetwo 
nearest control points of each curve. If 
wedon't want a crease between the 
curve, both tangent vectors have to 
point in the same direction. 

Theterrain demo generates evenly 
spaced x an d y val ues for a gri d of con - 
trol points Then I generatez values 
using a fractal terrain algorithm. After 
that, I do a post-pass on thepointsto 
move some of them to make sure that 
thiscondition holds. 

It's true that this continuity scheme is 
pretty limiting. For instance, if you have 
a patch surrounded by other patches, its 
entire border and all the points one in 
are completely defined by the other 
patches. That's every point in the patch! 
If we thinkof theterrain asachess- 
board, where each square is a patch, 
then after we define all the black-square 
patches, the white-square patches are 
already completely defined. This does 
mean that it's harder to have very local 
control over the terrain, but it'sstill 
quite possibleto make n ice-looking ter- 
rain. There are different, more relaxed 
continuity conditions, but they're more 
complex than what I've discussed. A 
description of these conditions can be 
found in Faux and Pratt's book in the 
ref eren cesattheendofthisarticle. 

With this implemented, we can gen- 
erate an array of control points and 
make patches from it, and know that it 
won't have any sharp seams or visible 



edges. Hypothetical I y, thiscould bea 
complete terrain system: a large array of 
textured, lit patches. What we haven't 
yet discussed though, arethespeed con- 
siderations. If we leave it like this, we'll 
bedrawing every patch, even those off- 
screen, and while the tessellation and 
drawing are fast for a single patch, if 
we're drawing a couple hundred patch- 
es, the frame rate will be abysmal. We 
need someway of drawing only those 
patches that are visible. 



Camera Frustum Culling 

Last month, I mentioned that the 
convex hull of a Bezier patch's con- 
trol points is a good bounding volume 
forthepatch. Therefore, if wemakea 
bounding box out of the control points 
by taking the minimum and maximum 
x, y, and z values, we can use the 
bounding box to cull the patch. We still 
need a way to tell whether the box is 
inside thecamera frustum, though. 

For my terrain demo, I use an object, 
ClipVolume, that takes information about 
thecamera and builds six Plane objects 
out of it: the left, right, bottom, top, 
near, and far planes. Planes are capable 
of telling you whether a point is inside 
or outside of them. Therefore, if all the 
points of a bounding box are outside 
any one of the planes, the box is com- 
pletely outside thefrustum. So, at the 
beginning of theframe, I build a 
ClipVolume out of thecamera by running 
through the patches and testing their 
bounding boxes against thedipVolume. 
Those that pass are drawn. 

It's simple, and definitely an 
improvement, but it'sstill not fast 
enough. Theproblem isthat wehave 
to touch every patch in theterrain 
each frame, so if we have an NxN grid 
of patches, therunningtimeof our 
culling isO(N2). That'snot ideal, 
because we don 't want the number of 
patch es to h eavi ly affect the frame rate. 
We need a way to find the visible 
patches without testing every single 
patch's bounding box. 



Quadtree Terrain Storage 

Asluck would haveit, there area 
number of ways to do this I used a 
quadtree data structure for theterrain 
demo. A quadtree is a tree where each 
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nod eh as four branches. In thiscase, the 
patch es are th e I eaf n odes. Th en , th e 
nodesat the next level of the tree each 
contain a 2x2 group of patches. They're 
contained in 2x2 groups by the nodesat 
the next higher level, and so on. 
Eventually, wehaveourtop node, 
which holds the whole tree. 

Each node contains a bounding box, 
which isthebox that containsall of its 
sub-nodes. Th is makes it easier for us to 
cull theterrain. We start at the top 
node, and if it's within thefrustum, we 
recurseon each of its four branches. If 
any nodeisever outside thefrustum, we 
just return and don't bother considering 
any of its sub-nodes, as they're all out- 
sidethefrustum, also. Thisway, wecan 
reduceour running time for culling an 
NxN grid of patches to O(lgN), which isa 
pretty tremendouswin overOIN^). 

To build one of these structures, we 
start with the patches, the leaf nodes. 
They have their bounding boxes. Then, 
out of each 2x2 block of patches, we 
build a higher-level node. The node's 
bounding box isthebox that contains 
each of the patches' bounding boxes; 
wejusttaketheminimum and maxi- 
mum X, y, and z from each of the four 
boxes to form its box. Once we have all 

• Farin, Gerald. Curves and Surfaces for 
CAGD, A Practical Guide. New York: 
Academic Press, 1997. 

• Faux, I.D. and M.J. Pratt. 
Computational Geometry for Design and 
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"Surface Simplification Using Quadric 
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GRAPH (1997): pp. 209-216. 

• Mortenson, Michael E. Geometric 
Modeling. New York: Wiley Computer 
Publishing, 1997. 

• Watt, Alan and Mark Watt. Advanced 
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Tlieoryand Practice. New York: ACM 
Press, 1992. 

• Thanks to Crack.com for releasing the 
full source, music, and texture library 
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• The terrain texture from Figure 7 was 
used from the Golgotha textures. 

The full source to the terrain demo is 
available from my web site at 
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of those second-level nodes, we use the 
same algorithm to generate third-level 
nodes from 2x2 blocks of second-level 
nodes. Wecontinueuntil weonly have 
one node, and that's our top node. 

Now we've got a smooth, textured, 
lit terrain culling quickly against the 
camera and drawing correctly. We're 
almost done. There's one last glitch 
that we have to take care of. 



Crack Fixing Revisited 

We fixed the cracking that occurs 
within a patch, but unfortu- 
nately, cracking also occurs along the 
edge between two patches when one 
patch istessellated to a different level 
from theother. Fortunately, inter- 
patch cracking isthe same kind of phe- 
nomenon as the internal patch crack- 
ing, and wecan usethe same approach 
to fix it. When we've finished tessellat- 
ing the patches we can see, wejust 
have each patch check its right and 
bottom edges against thepatches that 
share those edges. 

There is a catch, though. We have to 
make sure that we fix the inter-patch 
cracki n g before we d o th e i n tern al 
patch crack fixing. The reason fordoing 
so is a little confusing at first. We know 
that when we do the inter-patch crack 
fixing, wecan col I apse edge vertices 
into other edge vertices. Wealso know 
that when we fix theinternal patch 
cracking, wecan collapse non-edge ver- 
tices i n to ei th er oth er n on -edge verti ces 
or potential edge vertices. We know 
that internal crack fixing will never 
move an edge vertex, but it may col- 
lapse a vertex into an edge vertex. On 
theother hand, inter-patch crack fixing 
does move edge vertices. 

What doesthismean? Imaginean 
arbitrary vertex on the edge of a patch. 
We'll call it V. If we fix the internal 
cracks first, we might col I apse a vertex 
into V. Now when we fix theinter- 
patch cracks, we might collapse V into 
something else. When we do this, we 
leave the vertex that was collapsed 
into V hanging where V no longer is. 
This warps the patch and stretches a 
visible hole in the patch. 

If we fix the inter-patch cracks first, 
we'll collapseV into a corner. Then, 
when we fix theinternal cracks and we 
collapse that vertex into V, it will get 
sent to V'snew, correct location. 



Tilings Yet to Be Optimized 

This explanation iscertainly more 
difficult than the material present- 
ed in last month's article. Nonetheless, 
we'redone. Theterrain demo uses all of 
thethings we've discussed, from central 
differencing, texturing, lightmapping, 
and crack fixing to quadtree storage 
and camera culling. What's more, it 
looks good and runs fast. 

That's not to say that it's done. 
There's still plenty of work left to be 
done with theterrain system. For 
instance, the central differencing 
threshold can cause abrupt "popping" 
when the camera gets close enough to 
merit a higher level of detail. Perhaps 
a system where the change was grad- 
ual, using two thresholds, one where 
the detail began changing and one 
where it was done changing, could 
help. Linearly interpolating the size of 
the n on linearity vector over that peri- 
od would prevent abrupt changes in 
theterrain. 

Another aspect to work on is code 
optimization. While the algorithms 
themselves are fairly fast, the code is 
written to be legible instead of speedy. 
Hand-tuning some of the functions, 
such as the CentralPatchTessellator's 
tessellation function, could speed up 
the process. 

Furthermore, the current system isa 
bit of a memory hog. It uses 2D arrays 
for all theterrain, even though much of 
the data is unused, depending on the 
level of the tessellation and how much 
of theterrain isvisibleat any given 
time. A better memory management 
system could speed the system up by 
increasing cache coherency, and allow 
largerterrain sets within reasonable 
memory limits 

Of course there are aesthetic 
improvements and expansions that are 
begging to be made — a physics sys- 
tem, some wildlife, or some ponds 
would add quite a bit. The code is 
aval I able from my web site (see the ref- 
erence at left), and I invite all those 
who are interested to download it and 
make any improvements they can 
think of. 

Hopefully, between last month's arti- 
cle and this one, I've given a solid intro- 
duction to oneof the Next Big Things 
in 3D game engines. Send questions, 
comments, or cool modifications to my 
code, to meviae-mail. ■ 
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THIEF: The Dark Project 




btf Tom Leonard 

hief: The Dark Project is one of those games that 
almost wasn't. During the long struggleto store 



shelves, the project faced thethreat of cancellation twice. A 



fiscal crisis nearly closed the doors at Looking Glass. During 



one seven-month span, the producer, project director, lead 



programmer, lead artist, lead designer and theauthor of our 



renderer all left. Worse still, we felt a nagging fear that we might make a 
game that simply was not fun. But in the end, we shipped a relatively 
bug-free game that we had fun making, we were proud of, and that we 



hoped others would enjoy. 



The Concept 



The Thief team wanted to create a first-person 
game that provided a totally different gaming 
experience, yet appealed to the existing first-person 
action market. Thief was to present a lightly-scripted 
game world with levels of player interaction and 
improvisation exceeding our previous titles. The 
team hoped to entice the player into a deep engage- 



Tom Leonard wasthelead programma for Thief: The Dark Project, writing theAl and core architecture 
of the game. He lives in the Boston area. Tom has been at Looking Glassfor three-and-a-half years, prior to 
which hespent seven years working on C-i~i- development tools at Zortech and Symantec. Heiscurrently 
working on next-generation technologies, and can be reached at toml@lglass.com. 
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stealth is one of your best weapons in Thief. The game's designers made sure that expert players would have to make effec-\ 
five use of silent weapons such as the blackjack and the bow and arrow. 



mentwith the world by creating intelligible ways for the 
world to be impacted by the player. 

The central game mechanic of Thief challenged the tradi- 
tional form of the first-person 3D market. First-person shoot- 
ers are fast-paced adrenalinerushes where theplayer possess- 
es unusual speed and stamina, and an irresistibledesirefor 
conflict. Theexpert Thief player moves slowly, avoids con- 
flict, is penalized for killing people, and is entirely mortal. It 
is a game style that many observers were concerned might 
not appeal to players, and even those intimately involved 
with the game had doubts at times. 

The project began in thespring of 1996 as "Dark 
Camelot," a sword-combat action game with role-playing 
and adventure elements, based on an inversion of the 
Arthurian legend. Although development ostensibly had 
been in progress on paper for a year. Thief realistically began 
early in 1997 after the game was repositioned as an 
action/adventure gameof thievery in a grim fantasy setting. 
Up to that point we had only a small portion of the art, 
design, and codethat would ultimately make it into the 
shipping game. Full development began in May 1997 with a 
team comprised almost entirely of a different group of peo- 
plefrom those who started theproject. During thefollowing 
year, the team created a tremendous amount of quality code, 
art, and design. 

But by the beginning of summer in 1998, the game could 
not be called "fun," theteam was exhausted, and theproject 
was faced with an increasingly skeptical publisher. The 
Looking Glass game design philosophy includesa notion that 
immersive gameplay emerges from an object-rich world gov- 
erned by high-quality, self-con si stent simulation systems 
Making a game at Looking Glass requires a lot of faith, as such 
systems take considerable time to develop, do not always 
arrive on time, and require substantial tuningoncein place. 
For Thief, these systems didn't gel until mid-summer, fifteen 
months after the project began full development, and only 
three monthsbefore we were scheduled to ship. 

When the game finally did come together, we began to 
sense that not only did the game not stink, it might actually 
be fun. The release of successful stealth-oriented titles(such as 



Metal Gear Solid and Commandos) and more content-rich 
first-person shooters (I ike Half-Life) eased the team's concerns 
about th e market's wi 1 1 i n gn ess to accept experi mental game 
styles. A new energy revitalized theteam. Long hoursdriven 
by passion and measured confidence marked theclosing 
months of theproject. In thefinal weeks of the project the 
Eidostest and production staff joined usat the Looking Glass 
offices for the final push. Thegold master was burned in the 
beginning of November, just in time for Christmas. 

In many ways. Thief was a typical project. It provided the 
joys of working on a large-scale game: challenging problems, 
atalented group of people, room for creative expression, and 
theoccasional hilarious bug. It also had some of the usual 
problems: task underestimation, bouts of low morale, a 
stream of demos from hell, an unrealistic schedule derived 
from desire rather than reality, poor documentation, and an 
insufficient up-front specification. 

However, Thief also differed from a number of projects in 
thatittook risks on numerous fronts, risks that our team 
underappreciated. We wanted to push the envelope in 
almost every element of the code and design. The experi- 
mental nature of the game design, and the time it took us to 
fully understand the core nature of that design, placed spe- 
cial demandson the development process. Theteam was 
larger than any Looking Glassteam up until then, and at 
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times there seemed to be too many 
cooks in thel<itclien. Reacliing a point 
wliere everyone shared the same vision 
tool< longer than expected. A philoso- 
phy of creating good, reusable game 
engine components created unusual 
challenges that didn't always fit well 
with schedule and demo pressures. The 
many risks could have overwhelmed 
the project, if not for the dedication, 
creativity, and sacrifices of the team. 
Throughout the life of the project. 



more than 50 
people worked in 
oneway or anoth- 
er on Thief — 
some as part of 
the "Camelot" 
project, others as 
part of the 
Looking Glass 
audio-visual and 
technology sup- 
port staff, some as 
helpful hands 
from other 
Looking Glass 
projects. The core 
team consisted of 
a number of veterans 
of previous Looking Glass titles 
(Underworld I and II, System Shock, 
Flight Unlimited, Terra Nova, British 
Open Championship Golf, and the 
unpublished Star Trek: Voyager), as 
well as some new industry arrivals. The 
project had a number of very talented 
people and strong wills. Although it 
took some time for the team to unite as 
a tight-knit creative force, thefinal six 
months were incredibly productive, 
spirited, and punishingly fun. 



What Went Right 



1 Designing data-driven tools. Our 
# experience on previous titles 
taught usthatoneof the impediments 
to timely game development isthe 
mutual dependence of artists, designers, 
and programmers at every development 
stage. Oneof the development goals for 
the Dark Engine, on which Thief is 
built, was to createasetof toolsthat 
enabled programmers, artists, and 
designers to work more effectively and 
independently. Thefocusof thiseffort 
wasto makethegamehighly data- dri- 
ven and give non-programmers a high 
degree of control over the integration 
of th ei r work. M ed i a an d game systems 
were to be easily and intuitively 
plugged in and edited by theteam 
members respon si bl e for th ei r creati on , 
without requiring the direct involve- 
ment of programmers. 

The Dark Object System stood at the 
heart of our strategy. Primarily 
designed by programmer M arc "Mahk" 
LeBlanc, the Object System was a gen- 
eral database for managing the individ- 
ual objects in a simulation. It provided 
a generic notion of properties that an 



object might possess, and relations that 
might exist between two objects. It 
allowed game-specific systems to create 
new kinds of properties and relations 
easily, and provided automatic support 
for saving, loading, versioning, and 
editing properties and relations. It also 
supported a game-specific hierarchy of 
object types, which could be loaded, 
saved, and edited by designers. 
Programmers specified theavailable 
properties and relations, and the inter- 
face used for editing, using a set of 
straightforward classes and structures. 
Using GUI tools, the designers specified 
the hierarchy and composition of game 
obj ects i n depen den t of th e program- 
ming staff. In Thief there was no code- 
based game object hierarchy of any 
kind. 

Although the implementation of the 
system was much more work than we 
expected, and management of the 
object hierarchy placed significant 
demands on lead designer Tim 
Stellmach, it turned out to be one of 
the best things in the project. Once 
the set of available properties and rela- 
tions exposed by programmers was 
mature, the Object System allowed the 



designers to specify most of the behav- 
iors of the game without scripting or 
programmer intervention. Addition- 
ally, the relative ease with which vari- 
ables cou I d be made aval I abl e to 



designers in order to tweak the game 
encouraged programmers to empower 
the designers thoroughly. 

The second major component of our 
strategy was our resource management 




Hand-to-hand combat is sometimes necessary. 



POSTMORTEM 




system. The resource management sys- 
tem gave the game high-level manage- 
ment control of source data, such as 
texture maps, models, and digital 
sounds. It helped manage the game's 
use of system memory, and provided 
the data flow functions necessary for 
configuration management. 

Looking Glass's previous resource 
management system provided similar 
functionality, but identified resources 
by an integer ID and required a special 
resource compilation step. This tech- 
niqueoften required recompilation of 
the game executable in order to inte- 
grate new art, and required that the 
team exit the game when resources 
were published to the network. The 
new system referred to a resource by its 
file name without its extension, used a 
file system directory structure for 
namespace management, didn't leave 
files open while working, and required 
no extra compilation step. Developers 
simply dropped art into their local 
data tree and started using it. To 
expose art to the rest of theteam, lead 
artist Mark Lizottejust copied art into 
theshared project directories. 
Compound resources were treated as 
extensionsto thefilesystem and were 
built using the standard .ZIP format. 
This allowed us to use off-the-shelf 
tools for constructing, compressing, 
and viewing resource files. Thesystem 
facilitated content development by 
allowing programmers, artists and 
designers to add new data to an exist- 
ing gamequickly. 

The data-driven approach worked so 
well that through much of our devel- 
opment. Thief and System Shock 2 
(two very different games) used the 
same executable and simply chosea 
different object hierarchy and data set 



at run time. 

2 Sound as a game design focus. 
# Sound plays a more central role 
in Thief than in any other game I can 
name. Project director Greg LoPiccolo 
had a vision of Thief that included a 
rich aural environment where sound 
both enriched theenvironment and 
was an integral part of gameplay. The 
team believed in and achieved this 
vision, and special credit goes to audio 
designer Eric Brosius. 

As an element of thedesign, sound 
played two roles in Thief. First, it was 
the primary medium through which 
theAlscommunicated both their loca- 
tion and their internal state to the 
player. In Thief we tried to design Als 
with a broader range of awareness than 
thetypical two states that Alsexhibit: 
"oblivious" and "omniscient." Such a 
range of internal states would be mean- 
ingless if the player could not perceive 
it, so we used a broad array of speech 
broadcast by the Als to clue in the play- 
er. While very successful for humanoid 
Als, we feel the more limited express- 
ibility of non-human creatures is the 
heart of why many customers didn't 
like our "monster levels." 

Second, thedesign used soundsgen- 
erated by objects in the game, especially 
the player, to inform Als about their sur- 
roundings. In Thief, theAls rarely 
"cheat" when it comes to knowledge of 
tha'r environment. Considerable work 
went into constructing sensory compo- 
nents sufficientto permit theAlsto 
make decisions purely based on the 
world as they perceive it. Thisallowed 
us to use player sounds as an integral 



part of gameplay, both as a way that 
players can reveal themselves inadver- 
tently to theAlsand asatool for players 
to distract or divert an Al . M oreover, Als 
communicated with each other almost 
exclusively through sound. Al speech 
and sounds in the world, such as the 
sound of swordsclashing, were assigned 
semantic values. In a confrontation, the 
player could expect nearby Alsto 
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become alarmed by the sound of com- 
bat or cries for help, and was thus 
encouraged to ambush opponents as 
quietly as possible. 

In order for sound to work in the 
game as designed, we needed to imple- 
ment a sound system significantly 
more sophisticated than many other 
games. When constructing aTHiEF mis- 
sion, designers built a secondary 
"room database" that reflected the 
connectivity of spaces at a higher level 
than raw geometry. Although this was 
also used for script triggers and Al 
optimizations, theprimary roleof the 
room database was to provide a repre- 
sentation of the world simple enough 
to allow realistic real-time propagation 
of sounds through thespaces. Without 
this, it is unlikely the sound design 
could have succeeded, asit allowed the 
player and theAlsto perceive sounds 
more as they are in real life and better 
grasp the location of their opponents 
in the mission spaces. 

3 Focus, FOCUS, FOCUS. Early on, the 
# Thief plan was chock full of 
features and metagame elements: lots 
of player tools and a modal inventory 
user interface to manage them; multi- 
player cooperative, death-match and 
"Theft-match" modes; a form of player 
extra-sensory perception; player capaci- 
ty to combine world objects to create 
new tools; and branching mission 
structures. These and other "cool 
ideas" were correctly discarded. 

Instead, we focused in on creating a 
single-player, linear, mission-based 
game centered exclusively around 
stealth, with a player toolset that fit 
within the constraints of an extension 
of the Quake user interface. The notion 
came into full force with two decisions 
we made about seven monthsbefore 
weshipped. First, theproject was 
renamed Thief from the working title 
"The Dark Project," a seemingly minor 
decision that in truth gavetheteam a 



concrete ideological focus. Second, we 
decided preemptively to drop multi- 
player support, not simply dueto 
schedule concerns, but also to allow us 
asmuch time as possible to honethe 
single-player experience. In the end, 
some missionsdidn't achieve the 
stealth focus we wanted, particularly 
those originally designed for "Dark 
Camelot," but the overall agenda was 
the right one. 

4 Objectives and difficulty. One of 
# the Thief team's favorite games 
during development was Golden eye on 
the N64. We were particularly struck 
by the manner in which levels of diffi- 
culty were handled. Each level of diffi- 
culty had a different overlapping set of 
objectives for success, and missions 
were subtly changed at each level in 
terms of object placement and density. 
Relatively late in the development of 
Thief, we decided such a system would 
work well in our game. Extending the 
concept, we added a notion that as dif- 
ficulty increased, the level of toleration 
of murder of human beings decreased. 
We also allowed players to change 
their difficulty level at the beginning of 
each mission. The system was a success 
in two ways. First, it made clear to the 
player exactly what "difficulty" meant. 
Second, it allowed the designers to cre- 
ate a very different experience at each 
level of difficulty, without changing 
theoverall geometry and structure of a 
mission. Thisgave thegamea high 
degreeof replayability at a minimum 
development cost. 

5 Multiple narrow-purpose script- 
# iNG solutions. Although the 
Object System provided a lot of flexibil- 
ity, we also needed a scripting language 
to fully specify object behaviors. Rather 
than createa singleall-encompassing 
scripting system, we chose to develop 
several more focused tools for script- 
ing. Thistiered scripting solution 
worked well. 




In creating our core "high -end" 
object scripting technology, we want- 
ed to allow designers with moderate 
programming skill to create complex 
object behaviors easily. Scripts were 
event-driven objects attached at run 
time to game objects, and contained 
data, methods, and message handlers. 
The game provided a selection of ser- 
vices to allow the script to query the 
world state and the game object state, 
and also to perform complex tasks. 
Our goal was to createa scripting lan- 
guage that offered source-level debug- 
ging, was fast, and was dynamic. The 
solution was essentially C-H-in .DLLs, 
compiled by theC-H-compiler, using a 
combination of classes and preproces- 
sor macros to ease interface publish- 
ing, handledynamic linking, and pro- 
vide designers a clear programming 
model. Though used by both program- 
ming-savvy designers and program- 
mers, the fact that it was a real pro- 
gramming language prevented 
widespread use by all of the designers. 

M ost desi gn ers were i n terested i n 
customizing Al behaviors. For the Al 
we created a simpler scripting system, 
"Pseudo-scripts," that were implement- 
ed as properties within theObject 
System. Pseudo-scriptstook the burden 
of coding scripts off of the designers. 
The Al provided a stock set of triggers, 
such as "I see the player near an 
object" or "I see a dead body"; the 
designer provided the consequence of 
thetrigger. Each Pseudo-script was 
edited in a dialog box presenting para- 
meters to tweak the"if" clause of the 
trigger, and spacefor a list of simple, 
unconditional actionsto perform 
when the trigger fired. In thisway, the 
custom behavioral possibilities of the 
Al at any moment were described by 
theaggregate of Pseudo-scripts that 
were attached to that Al . This approach 
had three benefits. First, it was simple 
enough so that designers with no pro- 
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gramming experience were comfort- 
able using it. Second, it narrowed tlie 
range of triggers a designer could use to 
a good pre-selected set, rather than giv- 
ing them an open-ended system that 
might not haveworl<ed as well. Finally, 
when and how to evaluateAl triggers, a 
potential run-time expense if not care- 
fully constructed, could be custom 
built by a programmer. 

The final scripting system built into 
Th I EF was th e Tagged Sch ema system . 
When the game required motions and 
sounds, it requested them as concepts 
modified by optional qualifiers, rather 
than directly. For example, an Al who 
had just heard the player would 
request the concept "moderate alert," 
qualified with an optional tag like 
"-fsense:sound." A potential set of 
resources was then chosen using a pat- 
tern matcher; in thisexample, it would 
choose all samplesin that Al'svoice 
expressing a generic "something's not 
right," all samples expressing "I heard 
something fishy," but no samples 
expressing"! saw something fishy." 
From this set, the specific resource was 
then chosen using a weighted random 
selection. Thetables used were speci- 
fied by the designers using a simple 
language. Specifying motion and 
sound selection this way, designers cre- 
ated an interesting variety of random- 
ized environments and behaviors with- 
out changi ng the code of the game. 



What Went Wrong 

1 Trouble WITH theAI. If onething 
# could be called out as the reason 
Thief's gamepi ay didn't cometogether 
until late in the process, it would be 
theAI. TheAI as a foil to the player is 
the central element of Thief, and theAI 
wewanted wasn't ready until latein 
the spring of 1998. As lead programmer 
and author of the final Al, I take full 




respon si bi I i ty for th at. 

Th e ori gi n al Al f or Th i ef was 
designed by another programmer 
before th e req u i remen ts of th e revi sed 
stealth design were fully specified. Six 
months after it was begun, the project 
director and overseer of the system left 
th e team , an d th e most of th e program- 
ming staff was temporarily reassigned 
to help ship another gamethat was in 
trouble. During thefollowing months, 
development on that Al continued 
without any oversight and without a 
firm game design. Soon after, the pro- 
grammer working on theAI also left. 
Whilethecore path finding data struc- 
tures and algorithms were basically 
sound, thecodethat generated the 
pathfinding database was extremely 
buggy. The design of theAI decision 
process was geared towards an action 
fighting game requiring little designer 
customization, rather than a stealth 
gamethat needed much more cus- 
tomization. Even worse, thehigh-level 
decision process in theAI had drifted 
away from a rigorous design and the 
code was extremely brittle. Thewhole 
situation was a disaster. 

These might not have been serious 
issues, except for one key mistake: I 
didn't realize the depth of the problem 
quickly enough, and despite concerns 
expressed by programmer/designer 
Doug Church, I didn't act fast enough. 
I think highly of the programmer 
involved with the initial Al and wanted 
to avoid the natural but often misguid- 
ed programmer reaction within myself 
that I should just rewrite it my way. So, 
I took the position that, while buggy, 
th e system as a wh ol e was probabi y 
sound. Several months and many 
sleepless nights later, I concluded that I 
had been sorely mistaken. 

By November 1997, 1 had the basics 
of a new design and began working on 
it. But all work had to stop in order to 
pull together an emergency proof-of- 




concept demo by the end of December 
to quell outside concerns that theteam 
lacked a sound vision of the game. This 
turned into a mid-January demo, fol- 
lowed by an early February publisher 
demo, followed by a late February 
make-or-break demo. During thistime 
the only option was to hack features as 
best we could into theexisting Al. 
While better than losing ourfunding, 
constructing these demos was not good 
for the project. 

In theend, work on thenew Al didn't 
begin until mid-March. Despite the fact 
that our scheduled ship date wasjust six 
months away, we threw away four-fifths 
of our existing Al code and started over. 
After a hair-raising twelve-week stretch 
of grueling hours, theAI was ready for 
real testing. Had I committed to a 
rewrite two months earlier the previous 
autumn, I believe theAI would have 
been ready for real use three to five 
months sooner. 

2 An uncertain renderer. The pro- 
# ject was started because of the 
renderer, rather than the reverse. The 
basic core of the renderer for Thief was 
written in the fall of 1995 as an after- 
hours experiment by programmer Sean 
Barrett. During thefollowing year, the 
renderer and geometry-editing tools 
were fleshed out, and with "Dark 
Camel ot" supposed to ship sometime 
in 1997, it looked like we would have a 
pretty attractive game. Then, at the 
end of 1996, Sean decided to leave 
Looking Glass. Although he periodical- 
ly contracted with us to add features, 
and wewereableto add hardware sup- 
port and other minor additions, the 
renderer never received the attention it 
needed to reach the state-of-the-art in 
1998. The possibility that we might not 
have a point programmer for the ren- 
derer weighed heavily on theteam. 
Fortunately, Sean remained available 
on a contract basis, and other members 
of theteam developed sufficient 
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knowledge of the Tenderer so that we 
shipped successfully. In theend, we 
shipped a renderer appropriate for our 
gameplay, but not as attractive as other 
high-profile first-person titles. 

This may prompt thequestion of 
why wedidn't simply license a render- 
er. When the project started a few 
months into 1996, the avalanche of 
Quake licenses hadn't really begun 
and Unreal was still two years away. 
By th e ti me I i cen si n g was a vi abl e 
choice, the game and the renderer 
were too tightly integrated for us to 
consider changing. 

3 Loss OF KEY PERSONNEL AMID CORPO- 
# RATE ANGST BEYOND OUR CONTROL. 

M idway through 1997, Thief wasjust 
starting to gather momentum. We 
were fully staffed and thestealth design 
was really starting to get fleshed out. 
Unfortunately, Looking Glass's finan- 
cial situation was bleak. Few emotions 
can compare to the stress of heading to 
work not knowing who might be laid 
off, including yourself, or whether the 
doors would be locked when you got 
there. The company shed half of its 
staff in a span of six months, and while 
the active teams tried to stay focused, it 
was hard when one day the plants were 
gone, an other day the coffee machine, 
then the water cooler. 

Some of theTHiEF team couldn't 
continue under these conditions. We 
lost two programmers, including the 
former lead programmer, and a 
designer. When we were forced to 
close our Austin office, we lost our 
producer. Warren Spector, as well as 
some programmers who made valu- 



able technology 
contributions to 
our engine. All of 
these individuals 
are now on Ion 
Storm's Deus Ex 
team. Although it 
took some 
monthsto fully 
restore the spirit 
of the rest of the 
team, we held 
together and the 
company eventu- 
ally rebounded. 
Perhaps it 
bestowed a sto- 
icism that comes 
from knowing 
that however bad 
things might seem, you've already seen 
worse. 

4 Undervalued EDITOR. One of the 
# boils never lanced on the pro- 
ject was our editor, Dromed. Although 
it was sufficiently powerful and pro- 
vided the essential functionality we 
needed to ship the game, Dromed was 
a poorly documented and sometimes 
disagreeable editor. Dromed was first 
developed as a demonstration editor 
when thetarget platform of the game 
was DOS. As a demo, it never received 
the kind of formal specifications and 
designs one would expect for the cen- 
tral experience of the design team. Asa 
DOS application, it lacked theconsis- 
tent and relatively easy-to-use user- 
interface tools of Windows. An early 
mi stake was our failure to step back 
and formally evaluate the editor, and 
then rebuild it based on our experi- 
ence constructing the demo editor. We 
also should have designed a proper 
editor framework, and hired a dedicat- 
ed Windows user-interface program- 
mer to support it through develop- 
ment. In retrospect, thetimelost 
cleaning up the editor probably would 
have been saved on the back end of 
the project. 

5 Inadequate planning. Although it 
# is a cliche in the software 
industry to say our scheduling and 
budget planning were woefully inade- 
quate, the Thief project suffered greatly 
from this malady. There were several 
elements to our deficient planning. 

During "Dark Camelot," and continu- 
ing through thefirst half of Thief, we 
staffed theteam before the design and 



technology was sufficiently mature. In 
Thief, this led us to rush towardsfinish- 
ingthedesign, when wedidn't necessar- 
ily understand thedesign and technolo- 
gy. With insufficient specifications of 
both the code systems and mission 
designs, we ended up doing lots of con- 
tent that was essentially wrong for the 
game we were making. Code was written 
and spaces were built that weren't well 
d i rected to wards th e goal s of th e p roj ect . 

To make matters worse, we fai I ed to 
reassess core scheduling assumptions 
carefully once theschedulebegan to 
slip. Captivesof a seriesof unrealistic 
schedules, wedidn't leave enough 
time for the sort of experimentation, 
dialogue, and prototyping a project 
like Thief needs. Late in the winter of 
1998, many of our scheduling mis- 
takes had been corrected. Still, during 
the remainder of the project, the lega- 
cy of our earlier missteps required cut- 
ting missionsthat relied on technolo- 
gy wedidn't have, and reworking 
missions not focused on the core 
gameplay. 



Stepping Back from the Project 

THIEF was constructed asa set of 
appropriately abstract reusable 
game components designed for creat- 
ing object-rich, data-driven games. 
Although increasing the cost of devel- 
opment, this approach allowed 
Looking Glass to leverage various tech- 
nologies across d i sparate types of 
games, from the first-person action 
game System Shock 2 to our combat 
flight-simulator Flight Combat. In our 
next-generation technology, some of 
the systems, such astheAl and the 
Object System, will merely be revised, 
not rewritten. We intend to continue 
with this development philosophy in 
our future games. 

The next time around, our approach 
to constructing the engine will differ. 
Theengine will be scheduled, staffed, 
and budgeted asa project in its own 
right. The editor will be treated as more 
of a first-class citizen than wasthecase 
in Thief. Finally, a content develop- 
ment team will not be geared up until 
thetechnology is sufficiently mature to 
allow for an informed gamedesign 
process. 

Oh, and we'll get our schedules 
right — really. ■ 



game developer JULY 1999 



http://w WW. gdmag.com 



Backl 



SOAPBOX 



bif Maff Jo 



Don't Call Me "Audio Guy" 





hileattending the recent Game 
Developers' Conference in San 
Jose, I left each audio session feel- 
ing I ike the little teapot in that 



nursery rhyme, except that I was not 
short and stout, I was hot, steaming and 
pissed off. At one of the audio sessions, 
a well-respected audio designer began 
his presentation in a room full 
of budding young produc- 
ers with the classic, "I'm 
an audio guy and get 
no respect in this 
industry" inferiority 
complex. I've caught 
myself falling into 
this same trap in 
meetings and 
such, but 
propagating 
this 



think of someone who crawled off of 
the Van Halen 1984 tour. After destroy- 
ing the remainder of his hearing doing 
live sound to support hiscocaine habit, 
helanded a "dayjob" in the videogame 
industry. As you laugh, keep in mind 
that these people do 




impression 
throughout the 

gaming community is a big mis- 
take. We need to be proactive 
contributors to our community 
We need to assert confidence and power 
if we are going to be respected as equals. 

There are th ree major stereotypes 
that are keeping us down, yet we regu- 
larly embody them in public: 
The "Audio Guy." I am sick of people 
ref erri n g to th emsel ves as th e " aud i o 
guy." When I think of an "audio guy," I 



exist in 

our industry and they are 
all around you. (Oh, and by the way, 
we're not all guys Whilethetraditional 
linear media post-production scene has 
been a pickle convention for sometime 
now, thevideogameindustry contains 
some of the most talented female audio 
professionals in the world.) 



Computer gaming since 1983 and a charter member of theTed Nugent Bowhunting 
Association, Matthew Leejohnston is now a Senior Audio Designer at Microsoft. He 
can be reached at mattj@microsoft.com 



At a GDC audio session, I asked a 
question that made some people 
uncomfortable. I wanted to know why 
all current audio toolsare designed with 
control surface metaphors that were 
innovated in or before the late 1950s 
(such as volume sliders, piano rolls, 
knobs, and VU meters). An "audio guy" 
actually had theaudacity to tell me after 
the session that "you young guysjust 
need to sit back and wait your turn." I 
think that his main fear was that hewill 
someday have to become part program- 
mer and part audio guy. The truth is, he 
better, because the videogame of the 
future will demand complex and innov- 
ative audio systems that can't be 
designed by mere "code guys," 
because they don't understand 
thesubtleties of audio design like 
hedoes. But unlessheknows 
how to implement those ideas 
himself, hewill be back 
behind the mixing board 
faster than you can say, 
"Morevocalsin the 
monitor please." 
The Musician. At another ses- 
sion, one well known fig- 
ure within our community 
(who referred to himself as 
"TheAlmighty") 

^ addressed a room of 

more than 100 audio profession- 
als as "musicians." I 'm all for 
making music, but let'sface it, 
music isonly a small part of what 
goesinto a game these days Now, I 
understand that there are many people 
who represented themselves to their 
current employers as "computer 
experts" because they were wizards with 
the musician'sequivalent to a writer's 
word processor, the MIDI sequencing 
program. I also understand that some 
people fancy themselves rock stars 
because th ey make a good I i vi n g by 
orchestrating hundredsof little black 
boxes to do exactly what they want, 
when they want them to, and crank out 
the epic soundtrack to "Syncopation 2, 
TheAural Equivalent to Chinese Water 
Torture." However, most of us down 

Continued on page 63. 
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Continued from page 64. 

hereon planet Earth are also responsible 
for the other 80 percent of a game's 
audio, which includes thedialogue, 
sound effects, interface feed back, and 
so on. The longer we promote our- 
selves as musicians, the harder it's 
going to be for usto work our way into 
theupperranksof the team, which is 
where we must be if we are going to 
push audio design through to the next 
level. We need to be technologists, 
innovators, and sonic architects, not 
bong-smoking, noodle-headed, elec- 
tronic music geeks 
The Hollywood Big Wig. There is also a 
large number of linear media post- 
production sound designers who work 
amongst us Whilemany of these profes- 
sionals are extremely talented when it 
comes to complementing a fixed asset, 
Hiikeafilm, with sound effects, dialogue 
and music, they have limited expertise in 
creating an interactive soundscape. 
Programmers spend a great deal of ti me 



thinking universally about all aspects of 
the world they will becreating, and we 
also need to think well beyond thesonic 
content we create. 

There was at least oneGDC session per 
day that was sprinkled with a heaping 
dash of crow about how the "sound 
design didn'tturn outlikel wanted it 
to." The audio professionals always point 
theirfingersattheprogrammersand pro- 
ducers for thisshortcoming, which 
seems I ike a pathetic cop-out to me. 
When was the last time you provided a 
code sample with your spec? Every piece 
of mediaon your list should haveacom- 
pletedescription of how it interacts with 
all of itsdependencies: user input, artifi- 
cial intelligence, and so on. Leaveno 
stone unturned, do your damn home- 
work, take control of your world because 
if you don't, no one will. Waxing ecstatic 
about design concepts without being 
ableto provide thedeveloper with con- 
crete exam pies of how it can beimple- 
mented is negligent and can be offensive 



tosomeonewhohasspentthelast year 
thinking about everything but theaudio. 
Educate yourself on basic programming 
concepts, understand the relationship 
between thesound in thegameand 
everything it touches, and bea diplomat 
when it comes to working with the rest 
of theteam, because they areholding 
your design in their hands 

I believethatweareonly asinfluential 
as we want to be, because audio program- 
ming can bethe most emotional and 
subliminal (read: core) information a 
videogame can deliver. However, if we 
continue to play the role of the "audio 
guy," we will constantly be ruled by pro- 
gram managers producers and develop- 
ers, who all consider themselvesto be 
critical to the product's success. If you 
start showing up to thetable with the 
right cards in your hand, I can guarantee 
that your budget will grow, you'll break 
more bread, and the end product will 
bear a much greater resemblance to your 
original version. ■ 



